There weren't a lot of slaves in the field as far as I know. Many of these projects were on the frontier and it was all they could do to keep the army in the field and supplied. They weren't toting around hundreds of slaves. That being said, the status of a standard Roman solider wasn't much better than a slave but they were the ones doing the bulk of the work themselves.