If you think being a landlord is more hassle than it is worth then I challenge you to think again.
The life of a landlord does not need to be a drag. True, there are always extra obligations and risks you take on when you become a property investor. But then the same thing could be said of all business endeavours. By being clever with your time and more selective with (superfluous) tasks, you will be well on your way to keeping the extra stress under control.
Stop checking your (rental) bank account everyday - No no, I am not saying to let your tenant get away with not paying on time. That said, most people do not get their jollies on at the prospect of doing accounts and administration. If you have multiple tenancies, streamline your accounting practice by aligning all your tenancies so that all rents are due on the same day. Pick a day of the week when you do not have many other commitments so that you can spend an hour or so to go through your rent arrears protocol should you need to.
Stop calling your property manager all the time - Don't (over-) manage your property manager. Good property managers have systems in place to keep you in the loop. Most will furnish you with a monthly report on the state of the property as well as the rent account. If you feel the need to keep constant tabs on your property manager, you should start asking yourself whether you should be trusting them with your investment properties.
Stop scatter-gunning your maintenance/repair routine - Regular maintenance and occasional repairs are part-and-parcel to being a landlord. Instead of going into a blind panic each time your tenant rings about a leaking pipe, have a protocol in place so that you know exactly what you should be doing. A typical maintenance and repair protocol includes a contact list of trustworthy tradesmen, pre-existing email template for you to give adequate notice to your tenant, as well as an amount of buffer money sitting in your account ready to pay for the job.