Todoist/CSV to Anki Sentence Miner

This script automates the process of creating Anki flashcards from words saved in a Todoist project (or a CSV/text file). It fetches items, gets word definitions and example sentences using an LLM, and creates the notes directly in Anki via AnkiConnect.

Features

• Todoist Integration: Pulls tasks from a specified Todoist project.
• Flexible Word Parsing: Extracts words from task titles like {word}, English {word}, or just word.
• AI-Powered Definitions: Uses an LLM to get context-aware definitions for each word.
• AI-Powered Sentence Generation: Generates an example sentence for each word.
• Multi-language support: Build decks for any language (Estonian, Russian, …) via --learning-language; control the language of definitions separately via --instruction-language.
• Anki Card Generation: Creates cloze-deletion flashcards via AnkiConnect.
• Per-language subdecks: Every language gets its own subdeck — sentence-mining::<language> (e.g. sentence-mining::english, sentence-mining::estonian). The root deck is an empty container.
• Secure: Uses a .env file to keep your API keys safe.
• Automation-Ready: Can be easily set up with a cron job to run automatically.

Setup and Installation

Follow these steps to set up and run the project.

1. Clone the Repository

git clone <repository-url>
cd <repository-name>

2. Create and Activate a Virtual Environment

It's highly recommended to use a virtual environment to manage project dependencies.

# Create a virtual environment named 'venv'
python3 -m venv venv

# Activate the virtual environment
# On macOS and Linux:
source venv/bin/activate
# On Windows:
# venv\Scripts\activate

3. Install Dependencies

Install the required Python libraries using pip.

pip install -r requirements.txt

4. Set Up Environment Variables

The script loads your API keys from a .env file.

1. Create a .env file in the project root by copying the example file:

   cp .env.example .env

2. Open the .env file and add your API keys:

   TODOIST_API_KEY="YOUR_TODOIST_API_KEY"
   NEBIUS_API_KEY="YOUR_NEBIUS_API_KEY"
   

   The Nebius model is chosen at runtime with the required --model flag (see below) — there is no env var or code default.

Usage

Make sure Anki is open and running, then execute main.py:

python main.py --model <model-id> \
               [--source <todoist|csv|text_file>] \
               [--csv-file <path>] \
               [--text-file <path>] \
               [--tags <tag1,tag2,...>] \
               [--learning-language <code|name>] \
               [--instruction-language <code|name>]

Flags:

Flag	Default	Description
--model	—	Required. Nebius model ID (e.g. openai/gpt-oss-120b)
--source	todoist	Input source: todoist, csv, or text_file
--csv-file	—	Path to CSV file (required when --source csv)
--text-file	—	Path to text file (required when --source text_file)
--tags / -t	—	Comma-separated tags added to every note in the run
--learning-language	english	Language of the word and generated example sentence. Accepts ISO 639-1 codes (en, et, ru) or full names.
--instruction-language	english	Language of the definition/explanation. Accepts the same codes/names.

Examples:

# Default: Todoist source, English
python main.py --model openai/gpt-oss-120b

# Estonian CSV → Estonian deck + English definitions (default)
python main.py --model openai/gpt-oss-120b \
               --source csv --csv-file estonian_words.csv --learning-language et

# Estonian CSV with tags (e.g. a specific book)
python main.py --model openai/gpt-oss-120b \
               --source csv --csv-file csv-files/my-estonian-book.csv \
               --learning-language et \
               --tags "Source::MyBook,Topic::Fiction,Type::Book"

# Estonian CSV → Estonian deck + Russian definitions
python main.py --model openai/gpt-oss-120b \
               --source csv --csv-file estonian_words.csv \
               --learning-language et --instruction-language ru

# English book import with tags
python main.py --model openai/gpt-oss-120b \
               --source csv --csv-file my_book.csv \
               --tags "Source::MyBook,Topic::History,Type::Book"

Cards are created via AnkiConnect into sentence-mining::<language> subdecks (e.g. sentence-mining::english, sentence-mining::estonian). The root sentence-mining deck is an empty container.

CSV File Format

CsvSentenceSource skips the header row and infers columns purely from count, so header text is a label only — get the column count right and any header works:

Columns	Layout	Notes
2	word,context	Recommended. Tags come from --tags on the CLI, applied to every row. This is what every file in csv-files/ uses.
3	id,word,context	Adds a per-row id (logging only; CSV rows are never marked "complete").
4	id,word,context,tags	Adds a per-row tags column, comma-separated. Must be quoted if it contains more than one tag ("Tag1,Tag2") — an unquoted multi-tag cell overflows into extra columns and silently drops everything past the 4th.

word can be a single word or a multi-word phrase/idiom (e.g. beat the bushes) — no special markup needed, it's used verbatim. Quote any context cell that contains a comma. Example (2-column form):

word,context
gruesome,"If you use it without paying careful attention, the result can be gruesome."
horse sense,"But also the horse sense to work things out on the fly."

Automation with Cron Job

You can automate the script to run at regular intervals using a cron job.

1. Open your crontab file for editing:

   crontab -e

2. Add a new line to schedule the job. The following example runs the script every day at 7:00 AM.

   0 7 * * * /path/to/your/project/venv/bin/python /path/to/your/project/main.py
   

   Important:

   • Replace /path/to/your/project/ with the absolute path to this project's directory.
   • The command specifies the Python executable inside the virtual environment (venv/bin/python) to ensure the correct dependencies are used.

3. Save and exit the crontab editor. The cron job is now active.

Development Process

This script follows a layered architecture to separate concerns and make the code easier to maintain and extend (see AGENTS.md for the full breakdown).

1. Domain layer (domain/): core entities and interfaces — SourceSentence, SentenceSource, TaskCompletionHandler — independent of any specific data source.
2. Repositories layer (repositories/): thin wrappers around external APIs — TodoistRepository, LLMRepository (Nebius), AnkiRepository (AnkiConnect) — with retry logic via tenacity.
3. Data sources layer (datasources/): implementations of SentenceSource per input type — TodoistSentenceSource, CsvSentenceSource, TextFileSentenceSource.
4. Service layer: llm_service.py, anki_service.py, word_processor.py — business logic, injected with repositories rather than constructing them.
5. Composition root: main.py parses CLI args and wires the above together; config.py holds static config and secrets loaded from .env.
6. Anki integration: anki_service.py creates notes directly via AnkiConnect (no local .apkg packaging), with cloze-deletion and definition→word card templates, plus duplicate-note handling described below.

Anki Tagging System

The application implements a flexible tagging system for Anki notes, combining tags from multiple sources. This system utilizes a nested tag hierarchy (using ::) for better organization and leverages Anki's powerful filtering capabilities.

Recommended Tag Structure:

• Time: Year::YYYY (e.g., Year::2026), Month::MM (e.g., Month::01), and InstructionLanguage::<Language> (e.g., InstructionLanguage::English). These are all automatically generated.
• Source Type: Type::Book, Type::News, Type::Podcast, etc. (e.g., Type::Book from a CSV or text file, Type::Todoist for Todoist tasks).
• Specific Source: Source::BookName, Source::NewspaperName, Source::PodcastName (e.g., Source::Harry_Potter, Source::New_Yorker, Source::NPR_Podcast). This can be added via command-line arguments for batch processing.
• Subject/Domain: Topic::Tech, Topic::Finance, Topic::Literature, Topic::History, etc.
• Functional Tags (User-Defined): These tags describe how the card behaves or its status.
  • Check: For cards that might have a typo, an incorrect definition, or require manual review.
  • Idiom or PhrasalVerb: To categorize multi-word expressions.
  • Critical: For words or phrases that are essential to know (e.g., for work, an exam).

How Tagging Works:

• Combination: Tags are collected from script-generated defaults, data source metadata (e.g., Todoist task labels, CSV tags column), and command-line arguments (--tags or -t).
• Deduplication: All collected tags are combined, and duplicates are automatically removed.
• Hierarchical Format: Anki's hierarchical tag format (e.g., Parent::Child) is used for better organization.
• Benefits: Using a robust tagging system in Anki allows for flexible study. You can create "Filtered Decks" based on specific tags (e.g., to study only words from a particular book before a test) while keeping all your cards in one main deck for daily, efficient review.

Example Usage (Command Line):

python main.py --model openai/gpt-oss-120b --source csv --csv-file my_book.csv --tags "Source::MyBook,Topic::History,Type::Book"
python main.py --model openai/gpt-oss-120b --source text_file --text-file my_sentences.txt --tags "Source::Article_Title,Topic::Science,Check"
python main.py --model openai/gpt-oss-120b --source csv --csv-file csv-files/book-project-hail-mari-until-page62.csv --tags "Source::Project_Hail_Mary,Topic::SciFi,Type::Book"

Cron Job Setup Instructions

Step 1: Make the Bash Script Executable

First, you need to give the bash script execute permissions:

chmod +x /path/to/your/project/sentence_miner_todoist.sh

Replace /path/to/your/project/ with the actual path to your project directory.

Step 2: Test the Script Manually

Before setting up the cron job, test that the script works correctly:

/path/to/your/project/sentence_miner_todoist.sh

This should:

1. Activate your virtual environment
2. Run the main script with the todoist source
3. Generate the Anki deck
4. Log completion to cron.log

Step 3: Open Crontab Editor

Open your crontab file for editing:

crontab -e

If this is your first time, you may be asked to choose an editor. Select your preferred editor (nano is easiest for beginners).

Step 4: Add the Cron Job

Add one of the following lines to schedule your job:

Run Daily at 7:00 AM

0 7 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

Run Every 6 Hours

0 */6 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

Run Every Day at 10:00 PM

0 22 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

Run Twice Daily (8 AM and 8 PM)

0 8,20 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

Important: Replace /path/to/your/project/ with your actual project path in both places.

Step 5: Save and Exit

• If using nano: Press Ctrl+X, then Y, then Enter
• If using vim: Press Esc, type :wq, then Enter

Step 6: Verify the Cron Job

Check that your cron job was added successfully:

crontab -l

This should display all your scheduled cron jobs, including the one you just added.

Understanding Cron Syntax

The cron time format is: minute hour day month day_of_week

* * * * * command
│ │ │ │ │
│ │ │ │ └─── Day of week (0-7, both 0 and 7 are Sunday)
│ │ │ └──────── Month (1-12)
│ │ └───────────── Day of month (1-31)
│ └────────────────── Hour (0-23)
└─────────────────────── Minute (0-59)

Examples:

• 0 7 * * * - Every day at 7:00 AM
• */30 * * * * - Every 30 minutes
• 0 */4 * * * - Every 4 hours
• 0 9 * * 1 - Every Monday at 9:00 AM
• 0 0 1 * * - First day of every month at midnight

Troubleshooting

Check if Cron is Running

sudo systemctl status cron

View Cron Logs

On most systems:

grep CRON /var/log/syslog

Or check your project's log file:

cat /path/to/your/project/cron.log

Common Issues

1. Script doesn't run:

   • Verify the script has execute permissions (chmod +x)
   • Check that paths are absolute (not relative)
   • Ensure the .env file exists in the project directory

2. Environment variables not loading:

   • The script automatically changes to the project directory before running
   • Make sure your .env file is in the project root

3. Virtual environment issues:

   • Verify the venv exists at venv/bin/activate
   • Test the script manually first

Testing Cron Job Timing

To test if your cron job will run soon, you can temporarily set it to run in a few minutes:

# Get current time
date

# Edit crontab
crontab -e

# Add a test job that runs 2 minutes from now
# For example, if it's 14:30, add:
32 14 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

# Wait and check the log file
tail -f /path/to/your/project/cron.log

Disabling the Cron Job

If you need to temporarily disable the cron job:

crontab -e

Then add a # at the beginning of the line to comment it out:

# 0 7 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1

To completely remove all cron jobs:

crontab -r

(Use with caution! This removes ALL your cron jobs.)

Additional Tips

1. Email Notifications: By default, cron sends email on errors. To disable:

   MAILTO=""
   0 7 * * * /path/to/your/project/sentence_miner_todoist.sh >> /path/to/your/project/cron.log 2>&1
   

2. Multiple Environments: If you have multiple Python projects, each should have its own bash script pointing to its own venv.

3. Backup Your Collection: Notes are written straight into Anki via AnkiConnect (no intermediate export file), so back up your Anki collection itself (e.g. via Anki's built-in backups or exporting a deck package from within Anki) rather than looking for generated files in this project.