Hi testing folks,
a few day’s ago I stumbled over the term ‚unknown unknowns‘. I never heard about this term and I wanted to know more about it, driven by the question: Isn’t this usefull for testing?
The term was used by US Secretary Of Defense, Donald Rumsfeld in the year 2002 during a news conference (see wikipedia).
In my point of view it’s a little bit philosophic to talk about it but on closer examination we can find out that there are the following ‚combinations‘ concerning ‚knowledge‘ possible and that it all anyway fits together!
combination 1: The ‚known known‘. There is knowledge we know. 🙂
Examples concerning testing: There is the knowledge about requirements, the knowledge from any of our stakeholders about the testing project, perhaps the knowledge about customer wiches or for example allready known bugs and given test cases we know and finally the knowledge about how we have to do our work when we have any kind of restrictions or rules to follow. In my point of view all this stuff is the hard daily testing business when working as a tester in any company. For a passionate tester this is often a ‚boring work‘, but it’s work which has to be done anyway – maybee driven by our stakeholders, driven by test management, driven by release plannings, given test concepts, test tools and so on.
combination 2: The ‚known unknow‘. That means that we know that there’s something we don’t know! 🙂
Example from the testing wold: We know that there is a behaviour in the SUT which is in the eyes of a tester a missbehaviour. But we don’t know if this behaviour is realy a bug or not because e. g. it’s not part (against) any written down or spoken out requirement – simple not mentioned or not concidered!
For passionate testers this might be a more interessting part of our work, because we have to ask questions to our stakeholders (product owners, developers, etc. – people wo have an interesst about ‚their‘ SUT!).
combination 3: The ‚unknown known‘. This is knowledge which is availlable somewhere or known by someone. But ourselves don’t know about that knowledge! 🙂
What does that mean? An example: A person who is new to any kind of project might not have the knowledge which is needed to have access in that projekt. It takes time to get this needed knowledge.
What does that mean for a testing or software developement projekt? Also here an example: When we as a tester start from the scratch on a new testing project, we might not now any kind of test cases which are allready known by some other person. So there’s knowledge about the SUT which we as a new tester don’t have.
As passionate testers we might have the focus on learning all about that allready given test cases of our new SUT, of the corresponding existing bugs and problems in the project, the knowledge about thow he project team works or simple knowledge about our stakeholders! So this situation is in my point of view far away from a ‚boring testing job‘ because earning knowledge is allways not boring (when you are interessted in it 🙂 )!
Last combination and for me the most interesting one: The ‚unknown unknown‘
There’s no knowledge in general, neighter explored in science nor any person or instance on earth doesn’t know about that knowlege. 🙂
What does that mean for testing? In my point of view this is the most interessting part of any testing we can do. It can be a reason why we test! To test we must explore! The more we explore the system under test, the more questions we can ask! The more questions we can ask, the more answers we can get (from any stakholders, test oracles, spoken out and written down requirements, customers, collegues, etc.). At least we have to talk about the things we have found out and with that information we might get new informations about requirements! The mission of our testing can be to find out the ‚unknown unknowns‘ and change them to ‚known unknowns‘ and then with further questions, answers and discussions to ‚known knowns‘! -:)
The thing is: With ‚knwon knowns‘ we can create realy detailled test scripts which we can use e. g. for automated testing or e. g. as simple ‚learning objects‘ for new or other people who need them!
That might be the mission for us and also it’s a kind of ‚art‘ why and how we can create reals good test cases! Good test cases doesn’t grow on trees! And good test cases are in my point of view firstly allways the ‚unknown unknowns‘. They can not have much details in first drafts. The more we know from the ‚unknown‘ 🙂 the more we can add detaills to the those test cases. It’s a hard but a never boring way to find them. The best way we can do that is using exploratory testing methodes, or later combinations of allready ‚known‘ test cases (test scripts) and explorative testing methods which we can use when we need to ‚explore‘ something ‚unknown‘:-).
One thing I want to add: Also A big number of bugs will be found on the way to uncover the ‚unknown unknowns‘!
Another example from the testing world I want to add:
Imagine we work in a company as a tester and part of our testing mission is that there’s a need concerning something like a burden of proof. In this case we have to show how and why we have tested something. And this is only possible when we know what we do and already did in this specific testing! (we need ‚known knowns‘ as pre condition for that! 🙂 )
In this situation the important thing is: It’s not possible to get all ‚known knowns‘. (means it’s not possible to test everything!) But with having allready some ‚known knowns‘ (e. g. given good detailled and written down test cases or test scripts) we can make our way discovering some ‚unknowns unknowns‘ and finally make them ‚known knowns‘ too. Then we can minimize the problem we might have about the ‚burden of proof! At least we don’t need to have fear about that!
What do you think?
kind regards, Ralf