Reflections from TestBash Brighton – Being Brave

I’ve been a member of the online testing community for a long time, initially though Twitter by following some testing ‘names’ and then through the Ministry of Testing. A lot has been written about how welcoming the Ministry is and how they put inclusion and diversity at the forefront of their thinking and planning and I can only echo that. They have not only helped me improve as a tester, they have helped me improve as a person and had a massive and unexpected impact on me. They helped me be braver!

I’ve written poetry and parody lyrics to songs since I was young. These were always for me, with a couple of minor exceptions to my partner! You know, valentines and other mushy stuff… But something happened along the way and posting my first foray into testing related output (https://www.thebigtesttheory.com/blog/2017/11/26/if-ady-stokes-2017) in the club as well as my blog (which was also for me, never expected anyone else to read it) was a very big step for me. It took a long time for me to decide to share, then even longer to hit the Create Topic button. Fortunately there are some very kind people in our community who enjoyed it and encouraged me to do more.

At Brighton in my talk, Test all the Things with the Periodic Table of Testing, I had an irrational idea to create a song for people to join in and hopefully remember my table. I always get nervous before talking but it was several levels raised above normal thinking about singing (well, what I call singing) in public. Again I was encouraged by many but I need to shout out to Gem Hill and Emma Keaveny (our brilliant Essentials host) for their encouragement. To quote Emma;

Yes my friend, be brave! It’s the best place to be brave.

And it was! Despite my shaky voice, messing up the last chorus and going out of rhythm I did it! With more help from the amazing Gwen Diagram and João Proença through a Twitter request I had my support and the whole crown joined in. Warning, if you play this, you will hear my truly awful singing. No excuses, but it is a bit out of sync… honest… 😊

https://twitter.com/ministryoftest/status/1113369481596801025

Lyrics below if you want to see but the story isn’t quite finished…

Two days later I was even braver in the 99 second talks at TestBash. I read out my poem, Does it Work. The very first time I have ever read out one of my poems. I was shaking badly throughout to the point I nearly dropped the mic (not in a cool at the end way like Vern Richards (love him) suggested).

Being brave can sometimes have its rewards and mine was incredible. I’ve been a long-time admirer of Angie Jones and was fortunate to sit on the same table of her for TestBash. It literally made my week when I collapsed in my chair, stunned by the reaction and that I had actually beaten my fears, and Angie high fived me and said, ‘good job’. I have had it confirmed that despite being a 52 year old the correct label for my reaction was that I had a complete ‘fanboy moment’! I make no apologies for this, Angie is a legend, period.

If you take anything away from my ramblings please let it be, be brave. I am very confident in saying that whatever it is, whatever you want to try, we got you. Me, Ministry of Testing, the whole community. No judgement, just love. Be brave.

We Will Test You (music by Queen; lyrics by Ady Stokes)

Buddy you’re a visual heuristic

Describing the breath of the testing universe

You got elements in place

Paths you can trace

It’s something you can use all over the place, Singin’

We will we will test you (come on)

We will we will test you

Buddy you have different elements

Covering how you can test all the things

You now have a place

You can embrace

You can use this table all over the place

We will we will test you (with the periodic table!)

We will we will test you

Buddy you can scope your projects

Listing all the things that you want to test some day

No mud on your face

It’s no disgrace

To help you test the things give it pride of place

We will we will test you

We will we will test you

If – Ady Stokes 2017

‘If’ by Rudyard Kipling was written in 1895.  Here’s my version, if Rudyard were a tester. Please excuse the artistic licence! 

If – Ady Stokes 2017

If you can keep your head and explain the quality of the product that’s due, 
When all about you are losing theirs, and blaming it on you. 
If you can trust yourself when others are doubting you, 
But make allowance for their doubting too. 
If you are treated worse than you really should, 
Or there’s misunderstanding of what you said, 
And that the future doesn’t look too good, 
Or being told that your art is dead. 

If you can advocate your worth and believe in testing’s value, 
And think of every possible practical scenario that could transpire. 
If you can dream of all the things the customer might do, 
Then analyse the results to take your testing higher.   
If you can meet with Triumph and Disaster,
And treat those two impostors just the same.
If you can script, but not make scripts your master, 
And make discovery of truth your aim. 

If you can force your heart and nerve and sinew,
To serve your turn long after they are gone.
And so hold on when there is nothing in you, 
Except the Will which says to the team: 'Hold on!'  
And explain simply your deductive and inductive reasoning, 
That was led by curiosity and guided by your intuition.  
If you can hear bias arguments but smile and keep on listening, 
Then use soft skills and diplomacy to keep everyone on mission. 

If you can talk with crowds and keep your virtue,
Or walk with Management - not losing the common touch.
If neither foes nor loving friends can hurt you,
If all stakeholders count with you, but none too much.
If you can fill the unforgiving minute, (or 99 seconds)
With sixty seconds' worth of testing value. 
And find a great community who’s in it, 
To share their knowledge and is truly diverse too. 

If you can treat everyone equally,
No matter their difference to you. 
If you can treat their gender, origins and history, 
As valuable to offer a different view. 
If you can be an ally to all those who need, 
And defend their right to be just who they are. 
Support everybody’s rights to succeed, 
And mentor their journey so they may go far. 

If you can see scenarios that will make the testing great,
As well as the risks that others do not spot.
If you can see the value in which tests to automate,
And more, the value in which to not. 
If you can use tools to add value to your quest, 
But understand their value to aid and not replace. 
If you’re not afraid of change and always do your best, 
And can interrogate a database. 

If you can be the personas and advocate for all, 
And the voice of the customer in use and value too. 
And advocate and test for your product to be accessible to all. 
Explore to make discoveries because that’s what you do. 
If you accept that those discoveries, 
Will change what you thought you knew. 
Look for threats and system recoveries,
And help defend against those too. 

And then know all these things here, 
Are just a part of what you do. 
And be brave and show no fear, 
Accepting our learning will never be through. 
If your passion for testing means that you will never quit,
You will be here to the end. 
Yours is the Earth and everything that's in it,
And - which is more - you'll be a tester, my friend!

Session Based Testing, Exploratory Testing and my Questions technique

SB – Session Based Testing - 

Technique Element

Sub section – Approaches

Since Jonathan and James Back (satisfice.com/sbtm) documented their Session Based Test Management approach combining exploratory testing, risk management and ‘management’ oversight there has been a lot written.  Hopefully by now most people know the benefits of exploratory testing and some of the various methods of recording that activity.  In this article I hope to share a brief overview so we are on the same page.  A list of the main benefits and some minor drawbacks.  And finally the question technique I apply when using this in my day to day work.

Overview:

The Session Based Testing approach was developed for a project to allow their test team to ‘organise their work without obstructing the flexibility and serendipity that makes exploratory testing useful’.  They needed a way to keep track of what was happening, so they could in turn report back to a ‘demanding client’, while ensuring the time spent created the biggest return on investment. 

Essentially this is structured

exploratory testing

to help organise thoughts, capture questions and insights and allow rapid feedback.  Key elements to this approach include;

• Each session is chartered (associated with a specific mission)
• Uninterrupted (as much as is possible)
• A template is used to record the details of the mission and findings
• Reviewable ( a ‘report’ is produced to document findings and questions and the tester is ‘debriefed’) 
• Time-boxed (with flexibility but generally no more than 2 hours)

In my opinion, there are a number of flexible points in the approach and tips that are worth being aware of, especially if you’re doing this for the first time;

• I don’t think it matters if you call it a charter, mission or focus.  As long as you generally stick to your subject, although picking one might help when sharing for consistency.
• Interruptions should be avoided if possible.  On occasion I’ve shut down Outlook and put my headphones on for these types of sessions.  At one time I even had a red flag on my desk which indicated do not disturb unless it was urgent. 
• There are templates available or you can create your own like I did.  Again it’s useful for consistency to stick with one you’re happy with.
• Reviewable.  A lot of focus is on ‘management’ reviews but team, peer or even self is fine, as long as what you find generates actionable insights rather than getting filed away never to add value.
• Time-boxed.  If you start small with something very specific that’s a good way to get a feel for this technique and learn to focus.  I can sometimes be like the dogs in ‘Up’ and be distracted by squirrels!  Learn to note where the squirrels are and why you need to look at them later.

Question technique and template:

I admit that I often use this as a mental reminder, rather than something to populate, as my preference is to speak to a developer on my team immediately after a session to investigate or question.  (I don’t raise bugs I describe behaviour and in writing this, that’s probably what my next post will be on.  I’ll add a link to it on here when done.)  Only if this isn’t possible due to availability will I actually fill things in from the notes I have taken during sessions.  For me, this is a disposable document with a short shelf life used to capture, discuss, resolve (or not), and most importantly discard.

I’ve reproduced the template in bullet form rather than embed a PDF or word document, that way I hope it will be easier for you guys to take away and make your own.  When you get to the questions you might find, as I do quite often, you will remove some before you start as not applicable, or you won’t have filled some in when you’ve finished.  It’s supposed to be flexible like that but you should take a moment to understand why they are not populated or applicable to the session as that may prompt some other thoughts.

The template:

• The Basics: Date; App/function under test (brief description); any other useful information depending on your context
• Any dependencies vital to the testing (connections, files, data, hardware etc. this helps make sure you have them before you start)
• Any information that is useful such as material/learning’s from previous sessions, personas to use, environments, tools etc.
• Test strategy (a consideration of techniques you might use as a flexible plan is often more useful than no plan, but don’t be afraid to improvise as that’s half the fun and discovery may make your plan obsolete quite quickly)
• *Metrics (see rant at the end of this post)
• The questions: (with a brief reasoning for them)

o

What do I expect? (even if it is something brand new I always have some expectations)

o

What do I assume? (sets a context that I can query as I go)

o

Are there any risks I should be aware of? (to execution, the system, helps anyone else reading have context)

o

What do I notice? (behaviour; usability)

o

What do I suspect? (things that I feel, not always based on facts but that I don’t want to lose)

o

What am I puzzled by? (behaviour that doesn’t feel right)

o

What am I afraid of? (high priority concerns about the item under test)

o

What do I appreciate/like? (always good to have some positive feedback)

·

   Debrief (originally between the tester and a manager there’s a checklist of questions on

satisfice/sbtm

.  My version is more often a conversation with the developer with questions or queries, but can also be with the product owner or stakeholder depending on what I find/context. I’m not saying don’t do this, rather do it only where it’s going to add value.

This post is getting a bit longer than I’d hoped but I feel it’s important to summarise the benefits and possible drawbacks of using this method so there’s a balanced view.

Pro’s

Con’s

Allows control to be added to an ‘uncontrolled process’

Can be harder to replicate findings as full details are not captured

Makes exploratory testing auditable

As a ‘new’ technique it has to be learnt

Testing can start almost immediately

Recording exploratory testing (rather than brief notes) can break focus / concentration if you're more worried about doing it

Makes exploratory testing measurable through metrics gathered

Time is required to analyse and report on metrics

Flexible process that can be tailored to multiple situations

Time is required to discuss/give feedback to potentially the ‘wrong’ person

Biggest and most important issues often found first

Can help explain what testers do to clients, stakeholders and the uninformed

Given all the above, if you have to justify exploratory testing, (notwithstanding you should be looking for a new job!) then using session based test management either in its original form or some hybrid version could be the convincer you’re looking for.  In the table above, ‘management’ will generally only see the Pro’s column which covers a lot of the things ‘they’ will worry about.  But seriously, look for a new job!

*Metrics: I personally don’t think these are useful for virtually anything (oh, more controversy!), but if you absolutely have to report back to someone, a manager who knows little or nothing about what testing really is for example, here are some metric examples.  How much time you’ve spent such as start/end times on actually executing testing; blocked; recording findings; actionable insights; questions/queries; potential problem; bugs; obstacles; screen shots or some other method of recording or documents to show any issues to help replicate them.  

Things I learned at the North West Exploratory Workshop on Testing

Over the weekend of March 18/19 I attended NWEWT2 at the former Atlantic Tower Hotel, now Mercure in Liverpool.  Following on from its inaugural outing last year this deep dive conference focused on the theme of ‘Growing Testers’ with its two primary questions;

• As a Tester, how do you grow to keep up with the current trends in testing and development?
• As someone responsible for leading Testers, how do you help Testers grow?

Each attendee gave a talk on their thoughts, experience, models and ideas for growing Testers.  The audience was varied from those who regularly present at conferences to those with only a few years experience in software testing.  Following each talk it was ‘open season.’  The facilitator would note who had questions using a card system I hadn’t encountered before.  Green was a question following a talk, yellow was a follow up question during a ‘green’ thread and red if you had to say something immediately.  Fortunately there was only 1 of those and despite being taxing on our hard working facilitator it worked really well in not only ensuring conversations flowed with no interruptions, but also making sure everyone’s points were heard.  It also encouraged us to go deep into subjects and challenge each others ideas, cordially of course. 

My presentation focused on the creation and on-going development of Test Xchange, our internal testing community over the last year or so.  I offered our emergent model of a risk based participant built backlog and regular agenda items of lightning talks, discussions and testing challenges.  Given I heard about this opportunity at fairly short notice I must confess to a bit of a ‘cheat’ here as most of the material came from a scheduled talk I am giving at Test Atelier in Leeds on the 9^th of May.  So that was lucky!  (See related blog for more)

My main goals, besides sharing the great work our testing community has done in knowledge sharing, was to gain an understanding of how others were approaching tester development and bring back ideas we could use.  With coffee helping abate yawns from my 6am start driving from Yorkshire to Liverpool ready for a 9am start, we began.  So, what were the things I learnt over the two days? 

How does your garden grow?

There are so many ways to grow, gain knowledge and share information about testing.  The Testing Community is wide and diverse and there are many conferences, events and meet ups happening all round the country.  Online resources such as the Ministry of Testing are hubs of information not to mention the many talented individuals writing blogs.  Online magazines pass on knowledge and ideas while there are many excellent books on the subject.  YouTube is also a brilliant resource to watch the most ‘famous’ industry thought leaders share their views.  No matter your learning style there’s something for you out there.  All you have to do is find the time! 

I’m forever growing bubbles!

The Testing Community wants to help educators to pass on testing skills.  We all live in bubbles.  Family bubbles, social bubbles, community bubbles etc.  At the moment there’s an education bubble around software development that doesn’t have much room for the Testing Profession.  Course literature and Computer Science curriculums have little reference to testing as a discipline.  There were some passionate opinions expressed on reaching out to schools and Universities and offering the Testing Communities services to pass on knowledge of the ‘real world’ and testing skills, philosophies and passions.  The four-hour tester (http://www.fourhourtester.net/) is a project that focused on simple exercises to teach some of the skills and thought processes needed by testers.  I particularly like the Mary had a little lamb heuristic.  Give it a try.  The weekend testing community (www.weekendtesting.com) recently undertook a similar activity to create a testing syllabus.  Also well worth a read.  The question is now, am I brave enough to stand up in front of impressionable younglings and promote our profession?  I’ll get back to you on that one… What is for sure is that more efforts are needed to get out of our bubbles and into others.

How did I get here?

Very few people plan to be a tester.  Going round the table and aptly demonstrated by ‘Bullseye or The Testing Wheel’ (presentation available on slideshare.net by Ash Winter), career models are perceived as linear but very rarely are.  Despite our best plans be it through assignments, job change or sideways movement, most people swirl around quite a bit on their journey.  My own path to here has included roles such as logistics manager, auditor, director, business intelligence manager and test team lead to name a few.  While I had some version of testing in other roles it was by being asked to perform testing on a software system that peaked my interest. Then when I found out such an interesting and fun activity was a real full time job I was hooked.  What I didn’t realise is that there are so many similar stories of people moving in many varied directions on their road to becoming testers.  Maybe, like the saying about love, you don’t find testing, it finds you?

Mums have the best sayings!

You can’t stick your apples on other people’s trees!  Jit Gosai (test engineer at the BBC) talked to us about how he saw Mark Zuckerberg had floated Facebook and made billions at 27.  He was 27.  Did that make him a failure?  Of course not, but he vowed to do more, learn more and share what he found with others.  Later he was frustrated his sharing wasn’t landing how he’d like.  His mum told him about apples.  You can influence, you can give knowledge, you can lead, but you can’t make your ideas be someone else’s.  That doesn’t mean you should stop trying to share with those around you. No matter how annoying they are! 

So that’s some, but far from the only things learnt over an extremely enjoyable conference.  I’ve a decent sized list of people, models and sites I want to look into.  Another decent list of ideas for discussions, challenges and things we can do at Test Xchange.  Even some blog ideas.  And maybe, if I’m brave enough and survive Test Atelier, some more talks too.