Subconcious testing

I’ve finished reading Subliminal recently and it confirmed an idea I always had: guts in testing make a huge different. I’ve already talked about great testers being born instead of trained. Experience, study and failing to teach you new skills to achieve better efficacy, but most of the decisions I take during testing sessions are gut-driven (and experience driven as well, always remember previous shit!).

The best (or worst, depending on your side of the game) bug discoveries I’ve made were trying something off record purely based on my instincts. Building that load test focusing on the weird interaction between components because several parts use the same database, or just letting my inner annoying user take control of the keyboard and mouse. Something tells me that the calendar widget may easily fail, and the payment app is not going to correctly handle high load.

This makes some conversation with developers tense because they want a reason why we can’t go live get, and waiting for me up build a test that will check something that we don’t have any evidence yet that it might fail is not a valid reason. With time, they end up trusting my guts as much as me; but I find wise to hold it while you’re still building that trust, as it’s not wise to have your new team against you during the first weeks. When they see its value, they stop thinking that is a waste of time.

It also makes Quality Assistance methodologies trickier because teaching reasonings that are not fully conscious requires a deeper knowledge in the field. I like analysing my actions after doing them in order to understand the reasons and having an easier time explaining them.

But, if testing is about guts, how can we train it? For me, analysing my reasonings helps me identify what I correctly did and where to focus my efforts; and observing my colleagues asking for reasons make me understand new ways and approaches. It’s hard to learn from other’s experiences, but it’s still better than nothing. That’s why I prefer reading about testing cases and stories rather than studying. But there’s still a long road ahead so, give me a hint, how do you do?

May the force be with you

Gino

Cool automation tools

Few months ago I started gathering all testing articles and examples that I came across. I have a really awful memory, and discussing with my QA Lead about different approaches I could just remember some of the details, which led to awkward conversations; so starting to build my own “Testing Bible” sounded like an amazing idea. And so I did.

It’s actually a Google Doc where you can find chapters about specific testing methodologies (Rapid Software Testing, Quality Assistance), automation tricks, arguments to convince stakeholders and developers, a bullet point list of my achievements and some cool automation examples. Today I want to talk about the later!

I really love hearing about tools or techniques that organisation has developed to meet their really weird and niche testing needs. Google Testing Automation Conference is an amazing place to find some of them, as well as companies with tech blogs (although in some cases they think testing is not an enough appealing subject to talk about). So let’s talk about some of the ones I felt in love with!

Netflix Simian Army

This is bringing fault injection (https://en.wikipedia.org/wiki/Fault_injection) to the next level. In order to build a resilient system, Netflix realised that they need to be ready for errors. The best way the found to probe it was injecting errors in the pieces that are out of their control, so they understand what’s going to happen when those genuinely occurs. The talk is centred about the army of simians that they have causing AWS errors in production.

Things, like killing AWS instances or faking an entire region downtime on Amazon, are the duties of this monkey gang. They periodically trigger them on production to see the progress and be ready for when shit happens. Because we know that shit always happens. Part of the source can be found here.

Facebook’s test killer

As I’ve already mentioned, Facebook is known for not having Testing roles. They heavily rely on automation, although we know that flakiness and inconsistency is one of the biggest pains automating tests. That’s why they’ve come with a system where tests have to gain trust before the system will consider them as necessary; as well as a way to identify, heal and fix tests that cause more noise than benefits.

The system automatically marks the failed tests as “unknown value“, passing them through various states and identifying the responsible of it. It helps to review the code, and the system is the one disabling someone else’s test when it starts feeling flaky, so it’s not a “me against you” argument.

Generate tests from logs

Creating a complete suite of unit testing can be a daunting task. particularly when you scale up the product size. With this solution, they rely on an extensive logging and machine learning techniques to generate stateless and contextual assertions, building the skeleton of the functional tests.

You’ll still need some human intervention, but this is undoubtedly an amazing starting point, leveraging the most tedious part of the process. This is proper automation: something that assists us to achieve our goals!

Spotify’s Rapid check

Testing all the range of values in a unit test is impossible, that’s why Spotify has used the RapidCheck idea to help them build better coverage of the range. It randomly tests a broader set of values, setting a skeleton of important cases. Every tool that helps us get confidence on the solution faster is welcomed!

Do you know about another interesting testing tool? I’ve left some of them unsaid, but only because I love having pending subjects to talk about in future posts!

May the force be with you,
Gino

It’s always worse elsewhere

Yesterday I share a couple of beers with some old friends who also work in the Software industry. They’re working on a small Consultancy agency building websites using the same core, cutting most of the production costs as most of them use fairly similar implementations. Everything sounds right until they get surprised when I asked about their testing framework. “Well, before sending them to the client, we navigate through the site checking that anything breaks”. Fair enough, I don’t expect everyone to use Selenium. “The problem is when a client finds a bug. Most of them are part of the core so that bug is probably on every one of our projects, and we’re afraid of refactoring anything”. If the core is not a volatile piece of code, probably it’ll be a good investment to build some functional tests verification on top of your unit test. “Unit what?”. Bum. And then I remembered.

I was also part of a Consultancy brand (it was my first job in the industry). I remember the “we can’t afford to write the unit tests, our problem is really complex and we don’t have time, we’ll just verify it at the end”. I bought it. I truly believed that we were good enough that unit test was only a plus, and the projects were difficult and fast paced… NO WAY we could afford wasting time on tests! I remember some senior colleagues explaining that to me. Why would I don’t follow their example? They had way more experience than me, and they developed way faster. I wanted to be like them.

And I remember the struggles. I remember that every single bug fix carried weird and unexpected regressions. How the clients complained every week, and the constant fight about who is the responsible for the fix (Was it a bug, or a failure in the requirements?). The hellish deployments, the tedious manual verifications of just what was needed, the numerous problems with our version control tool as we always realised too late that the release didn’t contain that change.

I also remember the change. When I left the company and learnt something new in the next one. And then the next one. On my first day, one of my new colleagues was disturbed because I wasn’t writing unit tests. He didn’t even ask a reason, but I tried to convince him that they just slow my momentum. I remember his face. He just taught me how to make it easy, and handled me a copy of Clean Code. I had to pair with him and… Oh God, that was beautiful. He showed to me what TDD feels. And I started realising that, actually, the team didn’t spend that much time dealing with the horrors I was used to. It was so simple and beautiful.

That’s why I spent some time yesterday trying to show them the importance of this practices. Explaining to them why most of the companies they want to work in ask for TDD practices or writing functional tests. Exampling to them how they’ve been important during my projects. Giving them resources, guidelines and the chance to ask any question. But, obviously, I’m just a QA, so what the heck I’d know about coding.

Yeah, dear readers, it’s always worse elsewhere. That’s no excuse to not try to be the very best you, including the best professional you can be; but sometimes appreciating what you’ve learnt and why you’re doing it is needed. It is also important to identify who would use some help, and offer it. For me, life is how you feel sharing with the people you choose your knowledge, resources and smiles. I’ll talk in further posts about some of the

I’ll talk in further posts about some of the anti-patterns I’ve seen (and do) during my career, and some lessons I’ve learnt fighting them. But, for now,

May the force be with you,

Gino

Why being a tester spoiled my life

I truly believe that a great tester is born, as well as most other professions. Testing requires a strongly critical thinking, as well as perseverance and lateral thinking. There are some technical and products details that undoubtedly help in your daily work, but in my opinion, those skills don’t make a great tester on their own.

I’m not a great tester, and that’s what is driving me to understand, learn and practice how you become one. But I’m a critical thinker. I love breaking down situations too, in the calmest possible way, and identifying what is happening, which are the potential outcomes and how can it go wrong. I’ve been doing it my entire life, and probably that’s why I enjoy being a testing professional. But I can’t stop being a critical thinker in my personal life.

There are times, MANY times believe me, when spoiling yourself thinking what can go wrong will ruin an experience. There are times when understanding why you ended in some situation make it way less enjoyable. And, if you’re sharing it with someone, people will hate you when you try to anticipate the issues and prepare a plan B in advance. Some just want to do things, regardless of the result. A simple example is my peculiar relationship with food.

My mum in endocrinologist, and I grew up learning the calories of the food, which are their main nutrient and what is missing in any meal. And that has spoiled my life. I obviously eat junk food, and I enjoy eating colossal portions, but I can clearly point you how it will affect me, and what I’m doing wrong. So when I see someone eating a really big budget with doughnuts as buns, I just can think “SERIOUSLY??”, instead of enjoying it.

But peeps, if you’re cursed like me, remember that you can use it and become part of the testing community. Your colleagues will praise your ability to anticipate fires, and they’ll learn it after throwing away your recommendations; you’ll help people understanding why things didn’t work and you’ll show people that some details really matter.

This is why I’m still a tester. I spent part of my career pretending to be a developer, dreaming of being a designer and even thinking how would be producing. But I’m cursed, I was born to be a tester, and breaking things is SO funny.

May the force be with you,

Gino

Tester’s commitments v 2.0

I’ve already talked about James Bach’s tester’s commitments, but I have problems making that promise to my colleagues, as I don’t share some parts of them so I want to create my own version. I want to include Quality Assistance values, as well as some experiences; and keep them as short, simple and relevant as possible. These are my commitments:

  1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
  2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.
  3. I will assist you in the design of the product to ensure its testability.
  4. I will support you in any task to deliver a better quality product.
  5. I will provide guidance in your testing efforts, sharing my knowledge with you and helping with any tool or technique needed.
  6. I will learn the product quickly, and make use of that knowledge to test more cleverly.
  7. I will help you identifying important things to test first and try to find important problems first.
  8. I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product.
  9. I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer.)
  10. I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test.
  11. I will not carelessly waste your time. Or if I do, I will learn from that mistake.

I know that the list will evolve with time and experience, but these are the commitments I make today for you.

May the force be with you,

Gino

Gamificating testing

Gamification is one of those buzz words that you can easily find in articles, not only relating to the Software industry. There are numerous examples, both successes, and failures, but I want to talk about applying some gamification concepts to the testing world. I’ve never heard of any case, so I’ll talk about some things we tried and ideas I want to try in the future.

I’ll define gamification, so we are on the same page, as the application of gaming concepts to make a process or task more appealing to the practitioner. It usually involves a point system so the player can brag, some tier levels and perks related to them. In some cases, you don’t even have to give real rewards, like Reddit’s karma or StackOverflow’s reputation, as they rely on the power those systems have to gain recognition in very large social groups.

Obviously, they’re not appealing to everyone, although in my experience I’ve found some key factors that would dramatically increase its success like having a large enough group to contain some advocates of the game. There are always going to be detractors, but if you have enough people enjoying it the ball can keep rolling, and most of them are going to start realising the value. Keep the points meaningful and not easily obtainable, usually involving the players itself as the ones giving them. Allow players to brag about their points usually sharing a leaderboard. It’s usually better to keep rewarding the top players but avoid punishing the ones who doesn’t get involved.

How does this relate to testing? Well, as you know, we’ve been trying Quality Assistance for a while, and the biggest change is to convince everyone that Quality is a shared responsibility, and we expect some deliverables from them; for example developers should include their automated tests in the reviews, should spend some time testing a new feature, etc. One of the solutions we tried was including a point system where we reward the programmers who focus on quality, especially when they went above and beyond creating a testing demo, including non-functional tests, refactoring for testability or asking for help. We had a “testing padawan” in every team to understand how was it going. This way we could weekly showcase those coding rock stars: They felt rewarded, and we learnt which team should we focus our efforts on.

Other places where I think gamification would help us is in our user testing group. While we work on new products we seek for user feedback either for the prototypes and early builds. We find them among friends and family, power users or just random people. Ideally, if we implement a gamification method between our users, we can find the most engaged ones and reward them giving input for the new products, although this can be dangerous as most of the times new features are not targeted for these users. But when someone listens to you, it’s easier to get attached to a product. And that’s never a bad thing.

Can you think of any gamification example in the Testing industry? I’d love to hear it!

May the force be with you,
Gino

Developer archetypes

During my career, I’ve encountered numerous and very diverse professionals, which is one of the best perks of teamworking: you have to try really hard to not learn something from the experience, because learning how NO to do things is even more important than learning the best way. Reacting differently is always a factor that will empower the team as it’ll allow a wider expertise. But, obviously, there are some behaviours which help to build teams, while others make it challenging.

As most of my working hours are between developers (right now I’m the only QA for 15 programmers), I want to focus today on describing the different developer archetypes regarding Quality I’ve found, as well as sharing some tricks I’ve learnt dealing with them. Let’s start!

The “testing is not needed” one, which have two branches: “I’ve never seen proper testing, so I don’t see value in it yet”, typically found on junior profiles or professionals coming from experiences where no one cared about good practices; and the “My code doesn’t need to be tested” which can be even found on really senior profiles. For me, the wisest tip is focusing on them as few as possible.

When they start seeing the results from the others and realising that they’re getting behind, that they’re less shining, is the time to offer yourself to help them as much as needed. Your job is to make their work look awesome, don’t forget the commitments!

The “I don’t have time for testing” one, typical from junior profiles as well. They know that writing automated tests is valuable because… Well, probably because they’ve read that somewhere, and they know people expect it from them. Most of the cases they think there’s no time because they don’t know how you make them, so it’s a daunting task. The best that you can do is showing how easy is to get value from the tests. Handle them a framework with an example, pair during their first implementation and support them at the beginning.

If you see they become needy (what happens way less than I thought), gradually stop giving them that much support, coming from time to time to ensure they don’t get stuck. I’ve found that sometimes spending time explaining how this new skill is needed is they want to step up their career at some point towards “one of the big names” is really helpful.

The “I’m too biased to test” one, especially when you start talking about approaches like Quality Assistance and making them autonomous sharing the testing responsibility. They’ve heard that a developer can’t test, he’s biased, and that’s the perfect excuse to not even try it. The most successful way I’ve found to deal with this is focusing on the keener to test team members, sit with them and teach some tricks and examples. They love solving puzzles, and breaking systems!

When those colleagues are motivated enough, convince them to pair with the “biased” developers so they can test the new features together. Seeing a peer testing their code will show them that it’s possible to do it, and realising that when they review the dev on test feature finding way fewer bugs… Well, data is more convincing than words. And shame, more convincing than anything.

The “So, how can I test…?” one. This is the dream. These developers know what testing achieve, they understand that delivering a solution doesn’t mean anything unless it’s a quality one, they see value in the automated tests and prefer being the ones finding their shit. After seeing them work, and enjoying the results of their effort (fewer errors on their developments); they get your trust and you can focus the efforts on the rest of the Archetypes.

That doesn’t mean all the work is done. We assume most companies aim to have these developers, and sometimes you’ll be blessed with loads of them in your project. Now you can start working on what they’ve hired you to do. You discuss with them to build more testable products, review their work, build additional tools, assist them on the hard-to-test features, and experiment with new approaches. This is the dream. And that’s why I’m spending all my effort right now to convince as many developers as possible.

I don’t want to finish without naming one of my favourite archetypes: the “then, what are we paying you for?”. At the beginning, I wanted to answer them in a similar way but… wasting energy won’t take you anywhere. That’s why I keep it simple now. “My job is to review you until you become autonomous. Then, they can make me redundant or I can start focusing on the interesting stuff: doing whatever it takes to increase the Quality of your deliverables”.

Have you encountered different developer types?

May the force be with you,

Gino