Some of the biggest things that bug me as a user of software is how they handle errors. Apple has been known for silently handling errors and not telling users that something went wrong, this is bad. Another example of bad user handling is how Git Kraken handles theirs, they have a toast in the lower left that disappears after a duration of time. Both of these are really frustrating to the user of the software.
The best way to handle errors is to provide clear and copy-able error messages for your user. You could provide this using an alert, clearly visible logs, or a (persistent until dismissed) toast message in the corner. How ever you do this, provide a way that the user can take the error to Google afterwards. I prefer the alert method so that I am forced to acknowledge that an error has occurred.