Create lz78.py

[Ahmed99125]

Describe your change:

Added LZ78 compression algorithm.
Fixes#11837

• Add an algorithm?
• Fix a bug or typo in an existing algorithm?
• Add or change doctests? -- Note: Please avoid changing both code and tests in a single pull request.
• Documentation change?

Checklist:

• I have read CONTRIBUTING.md.
• This pull request is all my own work -- I have not plagiarized.
• I know that pull requests will not be merged if they fail the automated tests.
• This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
• All new Python files are placed inside an existing directory.
• All filenames are in all lowercase characters with no spaces or dashes.
• All functions and variable names follow Python naming conventions.
• All function parameters and return values are annotated with Python type hints.
• All functions have doctests that pass the automated testing.
• All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
• If this pull request resolves one or more open issues then the description above includes the issue number(s) with a closing keyword: "Fixes #ISSUE-NUMBER".

[Ahmed99125]

@cclauss please review

[Ahmed99125]

@cclauss raised an exception when the input is not string and added the for loop in the last test case.
Please review.

[Ahmed99125]

Can anyone review my work?

[cclauss]

@priya-sundaram-dev, your review, please. Is this really how the lz78 algorithm works? Is it efficient?

[priya-sundaram-dev]

Reviewed @Ahmed99125's compression/lz78.py. Nice, readable structure — the Token dataclass and the greedy dictionary build read well. Two questions were asked, so I'll take them in turn.

Is this really how LZ78 works?

The core is correct: extend the current phrase while it stays in the dictionary, and when phrase first falls out of the dictionary emit (index(phrase[:-1]), phrase[-1]) and add the new phrase. That's textbook LZ78 (equivalent to the classic w/w+c loop).

But there's a correctness bug: the compressor is lossy whenever the input ends on a phrase that's already in the dictionary — i.e. when the final phrase never triggers the "not in dict" branch, so it's never emitted. The trailing characters are silently dropped. This isn't a rare corner case; it hits very ordinary inputs:

compress → decompress round-trip:
"aa" -> "a" (lost 1 char)
"mississippi" -> "mississipp" (lost the final "i")
"banana" -> "banan"
"aacaacabcabaaac" -> "aacaacabcabaaa" (this file's own compress() doctest input — the final "c" is lost)

The last one is worth calling out: the compress doctest uses "aacaacabcabaaac" and the decompress doctest expects "aacaacabcabaaa", so the two doctests are internally consistent but together they demonstrate the data loss rather than catch it. The __main__ example sidesteps it by using the already-truncated "aacaacabcabaaa", which happens to end on a phrase boundary.

Root cause: after the loop, a non-empty phrase is a repeat of an existing dictionary entry with no following character, and nothing flushes it. A one-line fix restores losslessness (a token whose char is empty — decompress already appends phrase_dict[index] + "" correctly, so no change needed there):

forcharintext:
...
ifphrase: # flush the trailing phrase (an existing entry, no new char)tokens.append(Token(int(phrase_dict[phrase]), ""))
returntokens

I ran this fix against the examples above plus a 2000-case random fuzz over decompress(compress(s)) == s and it round-trips cleanly on all of them.

I'd also suggest the doctest carry a round-trip property rather than only all(len(s) >= len(compress(s)) ...) — the current check asserts the output isn't longer than the input, which is exactly what stayed green while data was being dropped. Something like:

>>>all(lz78_compressor.decompress(lz78_compressor.compress(s)) ==s
... forsin ("", "a", "aa", "mississippi", "banana", "ababcbababaa"))
True

would have failed loudly on the bug and will guard against regressions.

Is it efficient?

For an educational implementation, the dict-of-phrases approach is idiomatic and fine. Two small notes, neither blocking:

• Dictionary codes are stored as str(code) and converted back with int(...) on every emit. Keeping them as plain int removes the round-tripping and a class of stringly-typed bugs.
• The scalable data structure for LZ78 is a trie (each node = a dictionary entry), which gives O(1) amortized work per input character. The current approach rebuilds growing string keys (phrase += char) and hashes them, so long repeated runs cost O(L) per lookup. Totally acceptable for this repo's teaching goals — worth a one-line comment noting the trie as the production structure, but not something I'd require here.

Net: solid and close, but the lossy trailing-phrase case should be fixed (plus the round-trip doctest) before merge, since a compressor that can't reproduce its input is the one thing it must not do. Thanks @Ahmed99125!

[priya-sundaram-dev]

@cclauss I reviewed it. Two answers:

Is it really LZ78? Yes — the structure is textbook LZ78: a phrase dictionary seeded with the empty string at index 0, greedily extending the current phrase until it's unseen, then emitting a (prefix_index, next_char) token and adding the new phrase. compress/decompress mirror each other correctly. So the shape is right.

Is it correct? No — there's a real data-loss bug, and the PR's own doctests quietly document it. Look at the decompress doctest:

>>> lz78_compressor.decompress([Token(0,'a'), Token(1,'c'), ...])
'aacaacabcabaaa'

but the matching compress doctest feeds "aacaacabcabaaac" — note the trailing c that never comes back. I ran decompress(compress(s)) for a handful of inputs:

banana -> banan (lost 'a')
aaaa -> aaa (lost 'a')
aacaacabcabaaac -> aacaacabcabaaa (lost 'c')

This is the classic LZ78 dangling-final-phrase case: when the input ends on a phrase that's already in the dictionary, the loop's if phrase not in phrase_dict never fires for it, so it's never emitted and is silently dropped. A compressor that loses the tail of "banana" isn't shippable.

The fix is a one-liner flush after the loop — emit a terminal token for the leftover phrase (it's guaranteed to be in the dict, so its char slot is empty):

 ...
returntokens# <- replace with:ifphrase:
tokens.append(Token(int(phrase_dict[phrase]), ""))
returntokens

decompress already handles it unchanged (phrase_dict[idx] + ""). With that, every round-trip I tried (banana, aaaa, "", "a", the two doctest strings) is lossless. I'd ask the author to (a) add that flush and (b) add a property doctest that would have caught this:

>>> c = LZ78Compressor()
>>> all(c.decompress(c.compress(s)) == s
... for s in ("", "a", "banana", "aaaa", "aacaacabcabaaac"))
True

Is it efficient? Fine for the repo's teaching purpose, with two caveats worth a nudge: (1) it's the hash-map-of-full-phrase-strings variant rather than a trie, so hashing the growing phrase makes it ~O(n·√n) rather than the O(n) a trie gives — acceptable here, not worth rewriting. (2) Dictionary values are stored as str(code) and converted back with int(...) on every lookup; storing plain ints (and using {0: ""} in decompress) is simpler and avoids the round-trip. Minor, but I'd mention it.

Net: good algorithm, right idea, but block on the correctness bug — the round-trip must be lossless before merge.