5 The most common words, and a report

5.1 A function for the most common words

count gives the counts in no useful order. To find the most common words, tally must sort the counts, most common first, and keep the first few. A function, top, does it. It sorts by count, highest first, and then by the word, so that two words with the same count keep one order. Then it keeps the first n.

Apply the patch to src/tally/__init__.py, from the top of the project:

curl -fsSL https://github.com/rahuldave/tally/commit/72a0129dcee765132ad267788636ff4af9066369.diff | git apply --include=src/tally/__init__.py

The patch puts top after count and before main, with two blank lines around it.

A test says what top must give. It gives top three words with their counts, and asks for the two most common.

Apply the patch to tests/test_top.py, from the top of the project:

curl -fsSL https://github.com/rahuldave/tally/commit/72a0129dcee765132ad267788636ff4af9066369.diff | git apply --include=tests/test_top.py

The patch makes the file tests/test_top.py, with its one test.

uv run python -m unittest discover -s tests -q

The four tests pass.

Check your files against this move. The command prints nothing when they match, and otherwise shows what differs:

git check 72a0129dcee7

If you are stuck, this command puts the files of this move in place of yours. A file that you made yourself stays:

git restore --source f79588d3b046 --staged -- :/ && git restore --source 72a0129dcee7 --staged --worktree -- :/

@@ -11,6 +11,11 @@ def count(text: str) -> dict[str, int]:
     return counts
 
 
+def top(counts: dict[str, int], n: int) -> list[tuple[str, int]]:
+    "The n most common words, most common first."
+    return sorted(counts.items(), key=lambda pair: (-pair[1], pair[0]))[:n]
+
+
 def main() -> None:
     text = open(sys.argv[1]).read() if len(sys.argv) > 1 else sys.stdin.read()
     for word, n in sorted(count(text).items()):

@@ -0,0 +1,8 @@
+import unittest
+
+from tally import top
+
+
+class Top(unittest.TestCase):
+    def test_most_common_first(self):
+        self.assertEqual(top({"a": 1, "b": 3, "c": 2}, 2), [("b", 3), ("c", 2)])

5.2 tally --top N

tally prints every word of a text, and the corpus fills the screen. An option, --top N, makes main print only the N most common words, with top. Without the option, main prints every count, as before. main reads the option before the name of the file, so the command is tally --top 3 data/corpus.txt.

Apply the patch to src/tally/__init__.py, from the top of the project:

curl -fsSL https://github.com/rahuldave/tally/commit/c4b92c9bd158cd85d783cf17adc6bedc76c9ba0d.diff | git apply --include=src/tally/__init__.py

The patch writes main again, in place of the old one.

uv run tally --top 3 data/corpus.txt

The command prints three lines, the most common word first. They are 127 line, 126 class and 125 word.

Check your files against this move. The command prints nothing when they match, and otherwise shows what differs:

git check c4b92c9bd158

If you are stuck, this command puts the files of this move in place of yours. A file that you made yourself stays:

git restore --source 72a0129dcee7 --staged -- :/ && git restore --source c4b92c9bd158 --staged --worktree -- :/

@@ -17,6 +17,11 @@ def top(counts: dict[str, int], n: int) -> list[tuple[str, int]]:
 
 
 def main() -> None:
-    text = open(sys.argv[1]).read() if len(sys.argv) > 1 else sys.stdin.read()
-    for word, n in sorted(count(text).items()):
-        print(n, word)
+    "Count the words of a file, or of the standard input. With --top N, only the N most common."
+    args, n = sys.argv[1:], None
+    if args[:1] == ["--top"]:
+        n, args = int(args[1]), args[2:]
+    text = open(args[0]).read() if args else sys.stdin.read()
+    counts = count(text)
+    for word, times in top(counts, n) if n else sorted(counts.items()):
+        print(times, word)

5.3 A report, which just setup makes

The most common words of the corpus make a report, which others want to read without running tally. A recipe of the justfile, report, writes the ten most common words to build/top.txt. The recipe has two lines, and just prints each line before it runs it. The second line has two parts.

• mkdir -p build makes the folder build, and does nothing when the folder is there.
• uv run tally --top 10 data/corpus.txt counts the corpus, and keeps the ten most common words.
• > build/top.txt sends what tally prints to the file, in place of the terminal.

Apply the patch to justfile, from the top of the project:

curl -fsSL https://github.com/rahuldave/tally/commit/9e37b6eed29516f62bc176d1eaa66bf4e86683a6.diff | git apply --include=justfile

The patch adds the recipe report at the end of the justfile.

The report is an artifact, as the corpus is. git ignores the folder build/, so git holds the recipe, and not the report. just setup makes the report when the file is missing. test -f build/top.txt is true when the file is there, and || runs just report only when it is not.

The same patch adds a line to the recipe setup, after the line that makes the corpus.

A report from an earlier run stays, because git ignores it. So remove the report first, and then just setup makes it again:

rm -f build/top.txt
just setup
cat build/top.txt

The file has ten lines, the most common word first.

Check your files against this move. The command prints nothing when they match, and otherwise shows what differs:

git check 9e37b6eed295

If you are stuck, this command puts the files of this move in place of yours. A file that you made yourself stays:

git restore --source c4b92c9bd158 --staged -- :/ && git restore --source 9e37b6eed295 --staged --worktree -- :/

@@ -6,6 +6,7 @@ default:
 setup:
     uv sync
     uv run python scripts/corpus.py data/corpus.txt
+    test -f build/top.txt || just report
 
 # Count the words of a file
 run file:
@@ -14,3 +15,8 @@ run file:
 # Run the tests
 test:
     uv run python -m unittest discover -s tests -q
+
+# Write the ten most common words of the corpus to build/top.txt
+report:
+    mkdir -p build
+    uv run tally --top 10 data/corpus.txt > build/top.txt