Test Driven Development with ChatGPT

Introduction to Test-Driven Development (TDD)

Test-Driven Development (TDD) is a software development practice where you write a test before writing the code to fulfill that test. The core practice of TDD revolves around a short iterative development cycle that focuses on writing test cases for functional requirements before writing the functional code itself.

In essence, it works around these following steps:

1. Write a Failing Test
   Write a test case that describes a new functionality or improvement. This test should fail initially as we haven't written the code yet to provide that functionality.
2. Write the Code
   Write minimal code that will cause the test to pass. It doesn't need to be perfect - it just needs to be enough to pass the test.
3. Refactor
   Once the test is passing, you can rewrite or refactor your code without worrying about breaking anything. All transformations or changes to the code should be backed by the tests you've already written.

This approach provides multiple benefits: it encourages simple designs and inspires confidence. Regular application of TDD gives a more robust codebase and leads to high test coverage. It also ensures the code does exactly what it's supposed to do, as the functionality is defined via tests.

Some benefits of TDD are:

1. Improved Code Quality: TDD encourages writing tests before actual development, ensuring a high level of testing coverage and subsequently leading to a reduction in the number of errors or bugs in the finished product.

2. Easier Code Maintenance: The comprehensive suite of tests produced through TDD provides a safety net, allowing developers to make changes to the codebase confidently, knowing that test failures will immediately highlight any issues caused by an update or alteration.

3. Better Software Design: TDD encourages developers to consider how to structure code so that it's testable. This often leads to software that's better designed, easier to maintain, and simpler to understand, improving overall development productivity and code robustness. Enumerate the benefits of TDD, such as ensuring robustness, simplicity, and maintainability of code.

Introduction to ChatGPT

ChatGPT is a state-of-the-art language model developed by OpenAI, leveraging machine learning techniques to generate human-like text based on the input provided to it. It's powered by gpt-3.5-turbo, OpenAI's most advanced language model.

It's important to note that while ChatGPT can simulate an insightful conversation, it doesn't have beliefs or desires, doesn't understand the world or the text it generates, and doesn't have access to personal data about individuals unless explicitly provided in the conversation.

Understanding these two concepts, you can begin to visualize how TDD might be utilized in the context of testing ChatGPT, ensuring it performs optimally in the scenarios you require.

Setting Environment for Testing

Setting up an optimal environment for testing is a crucial step in Test-Driven Development. Here's how you can do it:

First, you'll need Python installed on your machine. You can download the latest stable version from Python's official website or use a package manager like brew (for MacOS) or apt (for Linux).

Next, set up a virtual environment using venv (available as part of Python 3.3 and later). This creates an isolated environment for your project, allowing you to install Python libraries without impacting your global Python installation. You can create a virtual environment with these commands:

bashCopy
python3 -m venv myenv
source myenv/bin/activate

Then, you'll want to install necessary Python testing libraries. pytest is a powerful and popular choice for Python testing:

bashCopy
pip install pytest

Thats it! Your're good to go now!

Writing the First Test

Let's start by writing a test for a simple Python function. Suppose we want to create a function that squares a number. We can start by writing a simple test case.

pythonCopy
def test_square_function():
    assert square(2) == 4

Here we’re telling pytest that test_square_function is a test to run. Pytest recognizes test functions and methods that start with test_.

You run your test with the pytest command:

bashCopy
pytest

Since we haven’t implemented the square function yet, running the test will fail. One of the principles behind TDD is that you write a failing test first before writing the actual code. Walk through writing the first test case.

Run it and show that it fails (as per the TDD process).

bashCopy
> pytest
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.11.5, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/silox/tmp/pytest_test
plugins: hypothesis-6.82.7
collected 1 item

test_square.py F                                                                                                                                                     [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________ test_square_function ___________________________________________________________________________

    def test_square_function():
>       assert square(2) == 4
E       NameError: name 'square' is not defined

test_square.py:2: NameError
========================================================================= short test summary info ==========================================================================
FAILED test_square.py::test_square_function - NameError: name 'square' is not defined
============================================================================ 1 failed in 0.05s ============================================================================

Now that you know how to write a test, you can ask ChatGPT to do it. Use all the clever prompt techniques to get your test. Remember, you don't have to get it right the first time.

Writing Code to Pass the Test

Now write code using ChatGPT using a prompt like this,

ChatGPT-PromptCopy
I want Python function to pass the following unit test:
def test_square_function():
    assert square(2) == 4

Run the generated code, with pytest For example, the generated output might be,

ChatGPT-AnswerCopy
Here is the Python function square() that should allow the unit test to pass:

def square(n):
    return n ** 2
This function takes an argument n and returns the square of n (i.e., n times n).

To explain, when the number 2 is passed to function square(), it will return 4. Hence, the assertion assert square(2) == 4 in your test will pass, as expected. Occasionally running tests on your code like test_square_function() is a good way to ensure your function is working as expected.

Copy the code to the same file(you can reorganize later), run the test and demonstrate a success.

bashCopy
> pytest
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.11.5, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/silox/tmp/pytest_test
plugins: hypothesis-6.82.7
collected 1 item

test_square.py .                                                                                                                                                     [100%]

============================================================================ 1 passed in 0.01s =============================================================================

TDD cycle with ChatGPT

When you're writing tests and code together, you have a chance to incorporate all the cases that your code must handle, be sure to include them in your tests. For example, if your square function must return a float value instead of a integer because the inputs are can possibly be floats as well, then write a test for it.

pythonCopy
def test_square_function():
    out = square(2)
    assert out == 4
    assert isinstance(out, float)

Again, you can ask ChatGPT to generate this test, with a prompt like this, if the test is complex,

ChatGPT-PromptCopy
Generate a pytest test function that tests the output of the function is a float

Now run the tests again, show that it fails,

shellCopy
> pytest
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.11.5, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/silox/tmp/pytest_test
plugins: hypothesis-6.82.7
collected 1 item

test_square.py F                                                                                                                                                     [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________ test_square_function ___________________________________________________________________________

    def test_square_function():
        out = square(2)
        assert out == 4
>       assert isinstance(out, float)
E       assert False
E        +  where False = isinstance(4, float)

test_square.py:8: AssertionError
========================================================================= short test summary info ==========================================================================
FAILED test_square.py::test_square_function - assert False
============================================================================ 1 failed in 0.04s =============================================================================

Ask ChatGPT to incorporate this case into account, on your old chat thread, ask ChatGPT to modify the code to satisfy this new test as well.

ChatGPT-PromptCopy
Modify this function to always return a float value.

Pick out the code portion of this prompt

pythonCopy
def square(n):
  return float(n ** 2)

Use this function and run the test. If it passes all your edge cases and scenarios, you're done, if it doesn't write a new test, point out that this certain case has failed to ChatGPT, use the new generated code and keep going. For example,

ChatGPT-PromptCopy
This function doesn't work when the input is a string.

Keep on improving your tests and asking ChatGPT for satisfying all the tests.

Expanding the Test Suite

It's important to note why we are writing new tests, what is the point of writing functions this way rather than just asking ChatGPT to write the function that satisfies whatever conditions you can think of.

The simple answer is, to not miss edge cases. If you have tests that assert some conditions then you can be sure that all the cases that you thought of were atleast satisfied by the function. Otherwise, as you keep iterating on the solution that ChatGPT gives, its possible that the edge case you solved in the first few prompts are missed by the solutions given by ChatGPT later in the conversation. This happens frequently when you're writing more than just a simple function.

Refactoring

Don't forget to refactor your code, it is also a crucial procedure in Test-Driven Development (TDD). It involves rethinking and revising the code to make it more efficient, maintainable, or readable, while ensuring that it still passes all its tests. Try to improve the software design and structure without altering its external behavior. It might include renaming variables for clarity, breaking down complex functions into simpler ones, or even rethinking the entire approach to solving a problem, ChatGPT solutions might not always be the best solutions. With every tiny change made during refactoring, it's essential that the test suite is rerun to ensure no broken functionality slipped in, thereby ensuring the software's integrity and helping catch any regression errors early on, preventing bigger problems in the future.

Conclusion

Test-Driven Development (TDD) combined with ChatGPT can be an innovative approach to software development, ensuring robustness and high test coverage. By embracing TDD, you can harness the power of ChatGPT while guaranteeing the code fulfills all your conditions and edge cases, resulting in high-quality software.

With that, keep your codes clean and your coffees strong. Happy coding!