No. Health and safety on site is a matter for the contractor, and the duty to reduce risks through design is a duty of the designer. Clients simply have to make sure that the initial project management arrangements are maintained. Do this by seeking assurance from the designer and contractor. For a non-notifiable job, simple enquiries will be enough to check that arrangements are in place to ensure:
adequate protection for the client’s workers and/or members of the public, if appropriate for the project
adequate welfare facilities have been provided by the contractor
good ongoing co-operation and communication between designers and contractors
project management arrangements agreed by the contractor to control key risks on site have been implemented
you do not need to get involved with the day-to-day running of the project. There is no obligation for a client to visit the site. If you feel you need advice then it is likely to be available from the competent person you have appointed under the Management of Health and Safety at Work Regulations 1999