This is a question faced by test team management across the industry. A tester has usually two major roles to play - as a checker and as an investigator.
As a checker you're verifying if something still works the way as it did before - you’re simply “checking” it works as it previously did. You as the test engineer are using existing knowledge usually captured in a test case or bug ticket (for example, in a test management tool like TestLink or QuickTest Pro, or a bug tracking tool like JIRA or BugZilla) and not really adding any new information to the current knowledge base. However, as an investigator you are attempting to discover new information about the application under test to help in risk-assessment of the project for both yourself and the stakeholders.
The focus of any automation effort is in the checks - you as the test engineer have typically already written the test case and manually executed it at least once; therefore the automation of that test case is checking to ensure nothing has broken after the initial release. This is essentially regression testing. Investigation is more difficult to automate as it does not involve following a set of pre-determined steps, and relies on the analytic and investigative skills of the tester during each click or swipe of the input mechanism.
For regression, good and thorough checks are a must-have and are best automated to get results quickly and repeatedly. This is the bread and butter of the automation engineer. Having said that, not every check benefits from automation: some of the checks are better done manually depending on complexity, privacy, or some other technical reason. For example, several years ago when writing some C# code for an automation framework that needed to confirm a particular popup, I spent hours attempting trying to get the code to recognize the child window…after many frustrated searches and ugly hack attempts, I simply decided that this part of the app would be tested manually. All kinds of technology these days suffer from similar fates - Flash, media files, and especially touch-driven devices, may all require manual approaches to test in their entirety.
The bottom line is that automation checks follow a pre-decided path and look for only those problems which have been specified in the code written for them. Although executed quicker and repeatable, it is a 2-dimensional approach to the problem of software quality assurance. Investigation by a live tester is slower, but they can unearth problems overlooked by the checks since the engineer is relying on their knowledge of the app, their general understanding of web technology, and sometimes their sheer good luck. Plus, while investigating the app for bugs the engineer is often designing the test cases and test plan in real time for future automation and manual re-use…you can easily see this is a more robust and 3-dimensional approach to maintaining high software quality standards.
There is clearly not a right or wrong answer to whether all testing should be automated, and each project approaches this discipline from different perspectives. My opinion is that only the time consuming checks should be automated, and oftentimes should include double checking the results of critical features and regression operations.
