As an employee, you should want to do you very best work every day, regardless of where you're sitting vis-à-vis your peers. However, if you think that working remotely means you can slack off more than your office-bound counterparts, working remotely is not for you. Some employer distrust comes from a history of dealing with workers who want to get away with doing less work than their peers. That has employers asking, "How can I trust that you're working if I can't actually see you working?” So work, and trust will follow.
June 29