Future Of Testing Recap
As a software tester, I started looking advice/resources and came across talks by James Bach. And he has really resonated with me. Other blogs and information I find lacking. I wasn't looking for tutorials on how to utilize tools, I wanted testing methodology and communication I wanted more of what James talks about in Future of the Software Testing Role. This is my opinionated summary of this talk, some of it is direct quotes some is my interpretation others are my closest approximation to what he said, but it all indicates the general time for when he said it:
2:45 — At this point James is setting the stage of where testing has been going and what has been driving it. He references the Agile movement and how this is a process set by developers and not testers, which is why you don't see testing as part of the process but just a side effect of the team doing work.
4:00 — He expresses that Agile's testing approach and his own are both in response to bad testing/testers. He is looking to improve the quality of the craft while feels Agile wants it gone.
4:30 — At this point James starts to make the case for specialization, but does really like the idea from Agile that people take on different roles, everyone should be testing. He later expands on this (1:07:00) saying he would like to see roles more like a villa.
7:40 — Developers have it easy, they don't need to explain to their manager what lines of code they were changing, their work is visible through artifacts of their changes (UI updates, reports, data); and even then all those line changes are saved historically in a CVS. Testers on the other hand can have a successful day of testing and not a single issue opened from it. The QA process is all about, by the time testing takes place the tester has nothing to show for his work. James makes a good point here that Managers are looking for testers to turn those testing actions into "legible" work. 10:25 "If you try to turn your work into little bricks called test cases you’re dumbing down your work. It does make it legible."
9:30 — As testers the work being done is abstract and it doesn't inherently produce something; "I produce knowledge, I inform you" that is what testing is, learning about the product and informing the team where the product stands.
12:00 — Jack-of-All-Trades master of none. This is often used to degrade the value of having a capability across different areas of expertise. James just points out that within your specialty you are also a generalist.
18:40 — Testing has some interesting characteristics which makes it hard to relate to from other fields: Accessible, Unbounded (24:45), Critical perspective (focusing on failures) (30:50), and questioning safe assumptions (53:10).
Testing has a initially very low bar of entry, it is an activity children do every day. Their testing the environment, boundaries, and social behaviors.
22:45 — James comments at this point that he believes that to some degree really good adult testers "never learned how to not say things that upset people, or don’t care as much."
24:50 — We as testers need to do a better job of agreeing to the possible, We are not going to “measure that we are for sure completed testing." There just is no metric to indicate complete, not code coverage (didn't measure data coverage).
27:25 — Testing is done because I've decided to stop working on it, and you absolutely need to make a decision to stop working on it.
Responsibility is a key point on testing, clearly this responsibility could be put on test leads, but even then every tester will need to make a determination that can be brought to the test lead as done.
28:25 — Acceptance criteria tends to be descriptions of the feature. This is great for developers because those things all need to be implemented. But rarely is it the criteria where when that behavior is displayed can the feature be shipped. "After I test this to a reasonable degree, which is impossible to define, we will accept it." That is the actual acceptance criteria.
38:10 — We all know that QA and testing are different things, all of the management are QA as they are there to assure quality. This is important because James is taking the position that he is a testing specialist.
I'm actually not sure this is completely accurate since James also does training for companies, where he is looking at the product and testing practices and guiding to improve that process, which I think would land him in quality assurance.
40:20 — Since James is trying to establish are market for a craft he emphasizes that while anyone can test/be a tester, a testing specialist has to meet higher expectations. Testing deeply and reliably, keeping in mind that testing deeply may not be necessary (43:30).
46:00 — As a specialist it should be possible to explain and defend the testing you've done. If someone challenges the work you did then you'll likely be excited because someone is taking interest in what you've done and now you have someones ear.
50:30 — This is very interesting, James says that he wants the tools to have good logging so he doesn't need to take notes and can just ask the software. Personally I get overwhelmed with excessive logging, but maybe with good logging I'd seek log parsing tools to help me.
James also emphasizes that he wants to be able to test the application at different levels, I think this is explicitly fighting against always seeking integration/UI testing.
53:10 — Another important aspect for a good tester is questioning safe assumptions. Safe assumptions are those which start making you look ridiculous because they don't fit with the view of the world. If you're running a decryption we could assumed that CPU fan speed isn't important, but since that has been used to break encryption its an assumption possibly important to question.
1:00:00 — It is important to understand that test coverage is never 100% and it can't be determined. There are a number of aspects to who or what is covering the test landscape. Of the possible tests humans will be doing the bulk of the testing (namely any machine testing is still actually directed human testing). A machine will be able to check over existing programmed validation, but will not create its own. On top of that there is a dependency between the testing done and the testing reported to have been done.
1:06:20 — As testers we a looking to understand the truth, not create it. We want to make that a process anyone can participate in.
1:07:00 — What can we do to make testing accessible to the team. In our job we hold a number of roles, one of which is testing, that is our Villa. How can we make the developers/PM's stay in our space comfortable so that they will want to come back?
1:20-00 — Where is testing headed?
1:30:15 — What makes a competent tester become competent?
- Seeking self respect
- Fleeing from pain
- Seek respect of people they respect
- Self study
- Deliberate practice
- Sharing experiences with other serious practitioners
- Making their own mistakes
- Experience in a related field
- Supervised experiences with feedback