The costs attached to your Python Strings

Python strings look innocent at first glance. They feel lightweight, read cleanly, and let you write things like this without a second thought:

s += "!"

But strings in Python are immutable. Pull on that string for a little longer, and the cost starts to add up.

Appending to a String, surely this is basic?

path = "/home/user"
path += "/documents"

While this looks like a harmless append, Python cannot extend path in place since the string is immutable. It must create a new string in memory, copy all the old contents, add "/documents", and point path to the new object.

This is linear in the length of the strings involved: O(len(path) + len("/documents")), which is an O(n) operation. Although the cost is trivial for small strings, repeat this over a large string or inside a loop, and the next morning you are complaining about how Python is slow.

Imagine you are reaching turn #413 of your long context chat and your agent harness has been rebuilding the entire conversation with string appends at every turn. That's the computational equivalent of 5 trips to the grocery store to buy 5 bananas.

Concatenating Two Strings?

system_prompt = "You are a helpful assistant."
user_prompt = "Explain Python strings."
prompt = system_prompt + user_prompt

Again, Python will first allocate enough memory for the result and then copy both strings into it. The cost is proportional to the total amount of text copied:

O(len(system_prompt) + len(user_prompt))

The effect becomes more noticeable when the result keeps growing.

📺 TV reference ↗The other Dr. Cooper

Strings in a loop ≈ Tangle

Naturally, this is where many of us start:

messages = [
    "User: Explain Python strings",
    "Assistant: Python strings are immutable...",
    "User: Why does that matter?",
]

conversation = ""

for message in messages:
    conversation += message + "\n\n"

But each iteration creates another string. If the result grows one character at a time:

The total amount of copying becomes:

1 + 2 + 3 + ... + n

In the general case, that grows quadratically: O(n²).

CPython optimizes some simple concatenation loops, so real-world timings may be better than this worst-case model. Still, repeatedly rebuilding a growing string is not a pattern worth relying on.

What would I do instead?

When all the pieces are already available, a better pattern is to use join():

conversation = "\n".join(messages)

Or, if you still need to depend on a loop when the pieces are produced gradually:

messages = []

while True:
    message = get_next_message()

    if message is None:
        break

    messages.append(message)

conversation = "\n".join(messages)

join() knows all the pieces before constructing the result. It can calculate the final size, allocate once, and copy each piece into place. With join(), the total cost is proportional to the size of the finished string: O(total characters) compared to the loop-append worst case of O(n²).

When I first started learning Python, join() always looked awkward to me. But I’ve come to appreciate it more now.

Other String Ops

Immutability doesn't make every string operation expensive.

• len(s) is constant time because Python stores the string length
• Indexing with s[i] is also constant time
• Slicing creates a new string: s[a:b] If the slice contains k characters, the cost is O(k).

Operations like replace(), upper(), and lower() inspect and return a new string, so they scale with the string length.

No Free Strings

Why are strings immutable in the first place?

Immutability is a useful design choice for Python strings. Because a string’s contents cannot change, strings can be shared safely, used as dictionary keys, and have their hash values cached. The trade-off remains the copy operation. Strings work well as completed values, but lists and other mutable buffers are usually better while text is still being assembled.

Can I make my strings mutable?

Python’s built-in str is immutable from the moment it is created; there is no option to reserve extra capacity or make it mutable ahead of time. For text whose final length is unpredictable, use the pattern shown earlier: assemble it in a mutable container, then create the final string once.

container = ['mutable', 'string', 'container']
# assemble text ... 
result = "".join(container)

For incremental, file-style text construction, Python provides StringIO:

from io import StringIO

buffer = StringIO()
buffer.write("hello")
buffer.write("!")
result = buffer.getvalue()

Build the text in a mutable in-memory buffer, then return it as an immutable string.

A small benchmark

Here is a small benchmark comparing repeated += concatenation with join() across increasingly large inputs:

from statistics import median
from timeit import repeat


def concatenate(parts):
    result = ""

    for part in parts:
        result += part

    return result


def join_parts(parts):
    return "".join(parts)


for size in (100, 1_000, 10_000, 100_000):
    parts = ["word"] * size

    concat_time = median(
        repeat(lambda: concatenate(parts), number=1, repeat=7)
    )
    join_time = median(
        repeat(lambda: join_parts(parts), number=1, repeat=7)
    )

    concat_ms = concat_time * 1_000
    join_ms = join_time * 1_000
    speedup = concat_time / join_time

    print(f"{size:,} parts")
    print(f"  +=       {concat_ms:7.3f} ms")
    print(f"  join     {join_ms:7.3f} ms")
    print(f"  speedup  {speedup:7.2f}×")
    print()

The exact timings will depend on the Python implementation, version, and machine. CPython can also optimize some simple += loops, so the benchmark may perform better than the worst-case model suggests. The comparison still illustrates the advantage of constructing the final string from known pieces.

You can run the benchmark below and see how it behaves as the input size changes:

Strings Attached

Most of the time, none of this is worth losing sleep about. A few concatenations here and there aren't going to ruin your code. It only becomes a real problem when a string keeps growing inside a loop. When you understand what Python is doing underneath, join() isn't the odd-looking syntax anymore.