Many years ago when the internet was ‘new’, I remember how easy it was to create a HTML web contact form.
It was quite a simple process.
A few lines of HTML code using a CGI or Perl script and you were done.
Unfortunately the internet has always been a place of scammers, spammers and hackers.
Web forms were no longer safe and were in fact quite an easy target for spam.
This problem has continued to this day and it is estimated that 14.5 billions spam emails are sent per day.
Whilst it’s not completely foolproof, the ‘Captcha’ method has been around since the late 90’s trying to reduce contact form spam.
In this article, I will give you all you need to know how to write test cases for how to test a Captcha.
Read on
What is a Captcha
The purpose of a Captcha is a mechanism of identifying and allowing a real human user access, and to keep automated bots and systems from submitting form data online. By using a series of verification techniques that only humans can conduct, the aim is to keep bots out. – Zaheer @ TestingChat.com
I’m sure there are plenty of better definitions, but that’s mine based on what I’ve experienced, seen and found from research.
What does CAPTCHA stand for?
“Completely Automated Public Turing Test to tell Computers and Humans Apart.”
It’s quite a mouthful.
I’m so glad I read this definition AFTER I came up with mine as it confirms exactly my thoughts.
What Types of Captchas Exist?
Early day Captcha consisted of a numeric maths problem.
This is where the user would be asked a question such as 14 + 1 and the user had to enter the answer.
Unfortunately, this wasn’t going to be a very good way to keep bot traffic out.
These then evolved into various types such as image, audio, video and puzzles to name a few.
This then evolved further to reCaptcha, reCaptcha v2/v3 and now reCaptcha enterprise. The brand was acquired by Google in 2009.
Below is a list of Captcha types with a description for each.
I’ve summarised research in an article called ‘Survey of Different Types of CAPTCHA’ which was published in the Journal of Computer SCience and Information technologies in 2014.
| Capture Type | Description | Pros | Cons | Example of Captcha |
| Text | A traditional Captcha consists of a text field which the user has to solve with simple questions. | Very easy to | 6 + 7 = ?14 – 2 = ? Which is a sports brand? IKEA, Toyota, Nike? | |
| Image | An image Captcha asks the user to ‘click an image like…’. | Hard for any artificial intelligence based systems to break pattern recognition. | If you are partially sighted, this may not be the best approach and therefore quite limited. I’ve also found that I don’t always get them correct the first time around. | In most cases, 2 random words that will be elongated, stretched or changed around so it is difficult for machines to read but easy enough for humans to read. |
| Audio | An audio captcha allows a user to hear the captcha code and enter it. | Ideal for the partially sighted. | May sometimes be unclear. | An image Captcha which also includes an audio button where you can hear the captcha code. |
| Video | A video will be played allowing the user to confirm a series of tags or words. | Good for those that may have hearing issues. | Not used very much. | |
| Puzzle | A puzzle Captcha asks a user to essentially complet a puzzle. | The answer that you are expected to put in may be a bit ambiguous. It’s unclear as to what the benchmark is as to what is correct and incorrect. This will have to be clearly defined in the requirements for it to be tested correctly. | Sliding an image to make it fit. | |
| ReCaptcha | This method of human verification includes analysing the user behaviour. | There are no text, images or puzzles to complete. | Doesn;t always get it right. | In most cases, the verification process will include a checkbox asking you to select if you are a human. Thats it. |
| Invisible reCaptcha | An automated way of detecting a human user without the need for verification. | No human interaction needed. | Can still be bypassed by advanced bots. | You probably don’t even know that an invisible reCaptcha has been generated. To test this, you will need a highly controlled testing environment where this can be tested. This will potentially be quite complex. |
How to Write High Quality Test Cases that even your Grandparent can run
Where would you use Captcha Functionality?
There are a ton of uses for this feature.
You’ll find Captcha being used pretty much everywhere online.
This is especially true where the application under test needs to verify the user accessing their system is a human and not one of those bad bots.
A really good example is when you are creating a registration page as this will use lots of personal data.
A login page is also a popular use of a Captcha as well as a reset or change password function.
Also, any type of functionality where a system can attempt a brute force attack may be a good option.
Have you ever noticed that if you login incorrectly a few times, you’ll be presented with a Captcha to confirm it IS you?
Examples of Test Cases for Captcha to Get You Started
Below is a list of test cases for a Captcha to get you started.
Generic Test Cases for Captcha Functionality
- Verify the Captcha is rendered in its entirety (check functionality and ideal for UI Testing)
- Verify that refreshing the screen generateds a random Captcha
Test cases for Text Captcha
- Verify that text image (captcha Code screen) is being displayed correctly
- Verify text can be read (functional and user experience check)
- Verify user is able to input data into Captcha field.
- Verify case sensitivity of captcha code
- Verify captcha validation failure when incorrect captcha code is entered
- Verify successful captcha validation when correct Captcha code is entered.
- Verify refresh button generates a new code
Test Cases for Image Captcha
Below are example for an image based captcha
- Verify images are all generate and rendered correctly
- Verify that the Captcha displays a meaningful message to the user on what they need to select.
- Verify the user is able to select 1 or more image(s) from the visual captcha that has been generated
- Verify user is able to select and continue
- Verify incorrect image selection and submission rejects the user.
- Verify when a user gets a submission incorrect, the system reloads the Captcha.
Test Cases for Audio Captcha
Just a note on audio Captchas. The audio feature in most, if not all cases will be integrated with a text Captcha otherwise its not fit for purpose.
- Verify Captcha User Interface has generated correctly.
- Verify the audio icon is displayed
- When Captcha has been generated Verify the audio icon can be clicked
- Verify that the Audio player and controls are displayed
- Verify each audio control functionality works (e.g. Play, pause, back, forward)
- Verify that selecting the ‘Play’ button generates the correct Captcha as per the screen.
- Verify audio Captcha is accepted
Test Cases for Video Captcha
I’ll be honest, I haven’t seen many video Captchas. They are few and far between compared to other styles of Captcha. If you do implement them then below are some test cases for you.
- Verify video captcha screen and all UI elements are generated correctly
- Verify that video button plays the video
- Verify video asks you a question related to the video
- Verify the video accepts the correct (related answer)
- Verify the video rejects an incorrect answer.
Here is a fantastic research paper on video captcha.
Test Cases for Puzzle Captcha
Whilst writing this article, I didn’t know that reCaptcha v2 was used as part of Google’s digitisation strategy so it could help its driving cars understand images and also its mapping software.
Below are test cases that may help.
- Verify that all puzzle images have been generated on screen
- Verify that all UI items and controls have been generated
- Verify the selecting the mouse button completes the correct action (you will to decide this based on what you’re trying to test)
- Verify that completing Puzzle Captcha allows you to progress to the next step
- this could be login authorisation etc.
- Verify that failing the puzzle captcha rejects the user’s actions and asks them to try again or the system takes another action.
Test Cases for ReCaptcha
How many of you knew that ReCaptcha that was acquired by Google in 2009 and helped to digitise Google books?
When I cast my mind back, I do vaguely remember that the ReCaptcha was helping to digitise books as this was mentioned somewhere.
I didn’t realise they were for Google though, wow!
Test cases are;
- Verify the reCaptcha screen has been generated on screen correctly
- Ensure the mouse screen is far away from the check box as possible and select the mouse button.
More on the official ReCaptcha site.
Considerations
Below are some of my thoughts and considerations.
Invisible ReCaptcha
This is quite a complex feature to test and possibly warrants its own article so I won’t cover it here.
You should however try and read up on it if your organisation wants to go down this route.
ReCaptcha enterprise
This is Google’s solution for organisations. I won’t be covering this here and may do in future articles.
Is a Captcha Effective anymore?
ReCaptcha v1 was decommissioned in 2018 as newer versions evolved.
A traditional Captcha generation program probably isn’t as effective as it once was with the rise of automated programs. However, they still remain quite popular.
Captcha technology such as reCaptcha v3 which claims to be invisible to a human is in full swing and more effective.
In a study published in the International Journal of Scientific Research in Information Systems and Engineering, all Captcha methods were deemed to be still prone to attack.
However, they are still effective in reducing bots and spam.
Innovations and Keeping Ahead of the Game
The Captcha business is a constant game of cat and mouse.
As web3 starts to dawn on us, we need to be aware of what the future holds and plan for the next phase of the evolution of the web.
Planning Your Test Scenarios in your Test Environment
When planning your test scenarios for your web application, there are a few key things to remember.
Regardless of the ‘Captcha type’, they work in almost the same way. So your tests should be quite similar.
- Ensure the “Captcha challenge” is generated fully on screen
- Ensure all the UI elements are visible as per your requirement specification documentation
- Ensure you can hear, read or see (video) the captcha.
- RUn a positive test to confirm the Captcha works
- Verify that it takes you to the correct destination page.
- Run a negative test using boundary value analysis. For example, using similar letters or numbers to try and confuse the system.
- Verify that it takes you to the correct destination page.
- Don’t forget Usability factors
- Don’t forget User Experience factors
Summary
As part of your software testing journey, writing test cases for Captcha functionality will need to be methodical, logical and well thought out.
Remember the purpose of a Captcha which is to keep the bots out and make it easy for humans to access.
Captchas should not be over complicated and not have a negative effect on the user experience.
With machine learning, artificial intelligence and natural language processing technology becoming more complex, Captcha technology needs to be able to accommodate changes, especially for those that use it in a negative way..
I hope this gives you some food for thought.
All the best with your testing.
