About

Sift is a notification hub. It puts what concerns you in one feed. You can read that feed and you can search it properly. If a source sends many notifications and most of them do not concern you, Sift also finds the ones that do. GitLab and Gmail are the two sources.

Why

There are two problems, and Sift answers both.

GitLab sends you an email about a change whether or not the change concerns you. Your mail client then holds a large pile of mail. Sift reads GitLab itself. It decides which items involve you. It shows you only those items.

The search of a mail client is poor. You cannot find a message in the pile again. Sift therefore takes every message from your mailbox. It makes no decision about your mail. It puts each message in the same feed as your GitLab items, and its search covers all of it.

What it tells you about

• Work that asks for you. Somebody assigns an item to you. Somebody requests your review. Somebody requests your approval. Somebody names you in a comment.
• Merge requests that still wait for you. Sift reads the current state of each one. It does not depend on a notification. You see a review request even when GitLab raised no to-do. You also see it when somebody dismissed the to-do.
• Discussions on an item you are part of. This covers a new reply in a thread. It also covers a new thread. You are part of an item when you are the author, the assignee or a reviewer. One comment of yours is also enough. Each thread gets one row. A thread with twelve replies stays one row.
• Commits on a merge request. This covers a merge request you review, one assigned to you and one you wrote. The row names the author of the branch. Your own push gets a row too, because the merge request comes back to your list whenever it changes and the row is what says what the change was.
• A merge request that somebody approves. The row names the person who approved it.
• A merge request that merges. The row names the person who merged it. The item leaves your list in a visible way.
• A pipeline that fails, and the pipeline that passes after it. Sift reads the pipeline of each merge request you review, of each one assigned to you and of each one you wrote. A failure is a row. The next pipeline that passes is a row as well, so you learn that it is fixed without opening it. A merge request you only commented on says nothing about its pipeline. A merge request that cannot merge is also a row.
• Every message in your mailbox. Sift makes no judgement about your mail. Each message becomes a row. This includes the mail that you sent. Each row opens that message in Gmail. Messages of one conversation make one entry.

Your own comments never notify you, and neither does your own approval. GitLab writes system notes about itself, and Sift ignores them. The note that records an approval is the one exception. Sift leaves out your drafts and your chats. A draft is not sent, and a chat is not mail. Sift also leaves out spam and trash, because Gmail leaves them out.

Status

These parts are built:

• accounts, and a connection to GitLab or to Gmail that you approve at that source itself
• every item in the list above
• the feed in day groups, with a colour for each type of event
• read and unread
• a Home page with one card for each source. A source you have not connected gets a card too, and that card connects it. The sidebar shows a source only after you connect it.
• a warning when a connection no longer works. An empty list then never looks like good news.
• a source that says Syncing now while Sift reads it, so a short list explains itself. For a mailbox it also names the month the reading has reached, because that takes hours.
• a button that reads a mailbox again from the beginning, which keeps every row and everything you have read
• a layout for a small screen. On a phone the sidebar becomes a bar along the bottom of the screen.

Read and unread behaves like this:

• If you open a row, Sift marks it read.
• Each row has a control. Use it to skip an item. Use it again to bring the item back.
• If an item moves again after you read it, it becomes unread again.
• A message you already read in Gmail arrives read.
• If you read a mail row in Sift, Sift marks that message read in Gmail. If you mark the row unread, the message becomes unread again in Gmail. This includes the button that clears every unread item at once. On a large mailbox, that button marks many messages read in Gmail in one action.
• The other direction works too. If you read a message in Gmail, the row becomes read here on the next sweep. If you make it unread there, the row comes back unread here.
• The counts above the feed are also the filter. All, Unread and Read are one click each.
• One button clears every unread item at once.
• The sidebar shows the unread count of each source on its icon. This tells you which source wants you, without a click.
• The browser tab shows the unread count after the name, as Sift (3), and on its icon. The count stays current while you look at a different tab. Your browser slows a timer in a tab you do not look at, so expect the count to change after approximately one minute. Sift must be open in a tab. It shows you nothing when it is closed.
• Sift can also make a short sound when something new arrives. It is off until you turn it on, in Settings under Sound. It sounds only while you look somewhere else, so it stays quiet while Sift is the window in front of you. Turn it on and you hear it once, which is how you know what it is. It has the same two limits as the count: a tab must be open, and a background tab is slow, so the sound can follow the arrival by approximately one minute.

Several events can happen on one merge request. Sift puts them in one entry, and you can collapse that entry. The list does not repeat the same title four times. A long list loads fifty entries at a time, and Sift asks the server for each page as you ask for it. A long history therefore does not make the app slower.

You can read the feed with the newest activity first. You can also read it with the oldest first. Use the second order to find the item that has waited longest.

Every page has a search field at the top. It searches every connected source, and it searches your whole history. It does not search only the tab you are on, and it does not search only the part of the list on screen. It forgives a typo. It forgives words in the wrong order. is:unread, is:mr, project: and from: make a search narrow.

For a message, the search covers more than the row shows you. The row shows the subject, the sender and the first line. The search also reads about the first thousand characters of the message itself, so a word from the middle of the first paragraph finds it.

The icon at the right of the field lists every prefix, with one line each on what it does. Hold the pointer over it, or reach it with the keyboard.

You can also narrow a search by date and by attachment. Write after:2026-08-01 for a date, or after:7d for the last seven days. The units are h, d, w, m and y. before: works the same way, so after:7d before:1d is one window. has:attachment keeps only the rows that carry a file.

Sift reads the name of every file that came with a message. It shows the first name on the row, and it searches all of them. Search for pdf, or for part of a file name, and you find the message that carried it. Sift ignores an image that a message draws inside itself, such as a signature, because nobody attached it.

Finished work does not drop out of the feed. Somebody completes a to-do, or a merge request merges. The row stays in the list. It turns grey and it says "done". The feed is your whole history. Read and unread is the only axis you filter it on. Finished work does not count as unread, because nothing waits for you there.

One thing does leave the feed. Sift never reads your spam or your bin, so a message you move to the bin in Gmail loses its row here. Take it back out of the bin, and the row comes back.

One source is not built yet. That source is Outlook. It needs a permission that the administrator of a company account must give, and that permission is the only obstacle.

Your Sift account is an email address and a password. A source gives Sift permission to read that source. A source never becomes your way in to Sift. Sift holds one account for each source, so one GitLab and one mailbox.

Run it

Docker is the only prerequisite. The Gradle wrapper installs its own Java 25 toolchain. The JDK on your machine does not matter.

make up        # builds and runs everything on http://localhost:7777

That is the whole setup of the app. On the first run Sift writes a .env file with a new encryption key. It waits for the database. It applies the migrations. It then serves the app.

To connect a source you must also register an application at that source. See Connect GitLab and Connect Gmail below. Do one, or both, or neither. The two are independent.

The search uses the fuzzystrmatch extension of Postgres, and a migration creates it. The bundled database allows this. If you point Sift at a database of your own, give its user permission to create that extension.

To work on the code, start the parts separately. Both sides then reload.

make db                      # just postgres
make backend                 # the backend from source, on http://localhost:7777
cd frontend && npm run dev    # vite, on http://localhost:5174

make help lists the rest. make logs follows the app. make stop keeps your data. make clean deletes the volume.

Connect GitLab

Sift asks GitLab for permission. You do not paste a token. First you make an application on your GitLab. You do this one time.

1. Open User Settings, then Applications, on your GitLab.
2. Give the application a name, for example Sift.
3. Set the redirect URI to http://localhost:7777/api/sources/gitlab/oauth/callback.
4. Select the scope read_api. Select no other scope.
5. Keep Confidential selected.
6. Save. GitLab then shows an Application ID and a Secret. GitLab shows the secret one time only.

Put those two values in .env, with the address of your GitLab:

SIFT_GITLAB_URL=https://gitlab.com
SIFT_GITLAB_CLIENT_ID=<the Application ID>
SIFT_GITLAB_CLIENT_SECRET=<the Secret>
SIFT_GITLAB_REDIRECT_URI=http://localhost:7777/api/sources/gitlab/oauth/callback

Run make up again. The GitLab card on Home offers Connect GitLab. Press it. GitLab asks you to approve. Sift sends you back to Home immediately and starts the first read behind you. The card says Syncing now while it happens, and it then gives you the counts. After that, Sift reads the source every five minutes. Settings offers the same button.

An administrator can prevent a person from making an application. If your instance does this, ask the administrator for an application. Sift has no other way to connect a source.

The access token expires after approximately two hours, and Sift renews it without your help. If the permission stops working, Sift says so on the feed itself. See If a connection stops working below. You can withdraw the permission on GitLab at any time. If you disconnect the source in Sift, Sift withdraws it for you.

Settings names the account that each connection belongs to, so you can see which GitLab user or which mailbox Sift reads.

You always know how current the list is. Each source tab gives the time of the last read, and a refresh control sits next to it. Each card on Home has the same control, next to the same time. Settings has the same function as a Check now button. All of them read the source immediately. None of them waits for the next pass. If a read fails, Sift gives you the reason.

The read_api scope is read-only, on purpose. Sift changes nothing on your GitLab. To mark a to-do done through the API, Sift needs the full api scope. That scope permits reads and writes across everything you can see. Sift gives you a link to GitLab for those actions instead.

Connect Gmail

Sift asks Google for permission. First you register a client in a Google Cloud project. You do this one time. Sign in to the Google account whose mail you want, then open console.cloud.google.com.

1. Make a project.

1. Open the project picker at the top of the page.
2. Select New project. Give it a name, for example sift.
3. Select Create. Then select the new project in the picker.

2. Enable the Gmail API.

1. Open APIs and Services, then Library.
2. Search for Gmail API. Open it.
3. Select Enable.

If you do not do this, the approval succeeds and every read then fails.

3. Configure the consent screen. Google renamed this area. Look for Google Auth Platform, or for APIs and Services, then OAuth consent screen. Both open the same pages.

1. Select Get started.
2. For App name, put Sift. For User support email, select your own address.
3. For Audience, select External.
4. For Contact information, put your own address.
5. Agree to the policy. Select Create.

4. Add the scope.

1. Open Data access, or Scopes on the older screen.
2. Select Add or remove scopes.
3. Filter for gmail.modify. Select the row for https://www.googleapis.com/auth/gmail.modify.
4. Select Update, then Save.

Google calls this a restricted scope. Sift needs it to mark a message read in Gmail when you read the row in Sift. It is the narrowest scope that can change a label. Sift writes one label and nothing else. See "What Sift reads and writes" below.

5. Add yourself as a test user.

1. Open Audience.
2. Under Test users, select Add users.
3. Put in the Gmail address you want to connect. Select Save.

If you do not do this, Google refuses the approval and shows "access blocked".

6. Make the OAuth client.

1. Open Clients, or APIs and Services, then Credentials.
2. Select Create client, or Create credentials, then OAuth client ID.
3. For Application type, select Web application. Give it a name, for example Sift local.
4. Under Authorised redirect URIs, select Add URI. Put in exactly http://localhost:7777/api/sources/gmail/oauth/callback.
5. Leave Authorised JavaScript origins empty. Sift never calls Google from your browser.
6. Select Create.

The redirect URI must agree with SIFT_GMAIL_REDIRECT_URI character for character. A final slash is a difference. Google permits http for localhost only, so this address needs no certificate.

7. Put the two values in .env.

SIFT_GMAIL_CLIENT_ID=<the client ID>
SIFT_GMAIL_CLIENT_SECRET=<the client secret>
SIFT_GMAIL_REDIRECT_URI=http://localhost:7777/api/sources/gmail/oauth/callback

Run make up again. The Gmail card on Home offers Connect Gmail. Google shows the message "Google has not verified this app". Select Advanced, then Go to Sift (unsafe). That message is correct. Nobody verified this app, because the app is yours. Sift sends you back to Home immediately and starts to read your mail behind you. A large mailbox takes a long time, and the card says Syncing now while it happens.

The seven-day rule

While your app stays in the Testing state, Google cancels the renewal token after seven days. Sift then shows Gmail as a connection that no longer works, and you connect it again. This is not a fault in Sift.

To prevent this, open Audience and select Publish app. The state becomes In production, and the renewal token then does not expire on a timer. You still see the message about verification, and a limit of 100 users applies. Neither one matters for your own instance. Verification is necessary only to remove that message for other people.

What Sift reads and writes

Sift reads all of your mailbox. The search finds only the messages that Sift read, and the search is why Gmail is in Sift. A limit on how far back Sift reads is thus a limit on what you can find.

The first read takes the newest messages. Each read after it does two things. It takes the messages that arrived since the last read. It also takes one more group of older messages. Sift thus moves back through your mailbox until it gets to the first message. A large mailbox needs many reads. You can use Sift while this happens, and the Gmail page tells you that older mail is still on its way. It also names the month the reading has reached, and that month moves with every read. The card keeps Syncing now for the whole of it, and gives you a time only when Sift has read your mailbox back to its first message. If several reads in a row reach nothing older, Sift says so. Google is usually refusing the pace, and a longer SIFT_SYNC_INTERVAL clears it.

Sift reads a message once and never looks at that message again. So a message read before a version of Sift that keeps something new carries only what Sift kept at the time. Read it all again in Settings answers this. It reads your whole mailbox from the newest end, and it keeps every row and everything you have read. Use it if has:attachment finds fewer messages than you expect, or if a search inside your older messages finds nothing. It takes as long as the first read did.

Sift usually shows many more items than Gmail shows you. There are two reasons. Gmail counts a conversation as one line, and Sift makes a row for each message in it. Gmail also opens on your Inbox, and Sift reads all of your mail. To compare the two numbers, use All Mail in Gmail and not the Inbox.

Each message costs one request to Google. If you make SIFT_SYNC_INTERVAL very short, Google can refuse the requests while Sift reads a large mailbox. Use a longer value until your mailbox is complete.

Sift writes one thing to your mailbox: the UNREAD label, which is what makes a message read or unread. It does this only when you read a row or mark one unread in Sift.

Sift sends no mail. It writes no other label. It deletes nothing, and it gives you no way to delete anything. The gmail.modify scope permits more than this, because Google has nothing narrower that can change a label at all. You can withdraw the permission in your Google account at any time.

If you disconnect Gmail in Sift, Sift withdraws the permission for you. It also removes the rows of that mailbox. Connect it again, and Sift reads the mailbox from the beginning, which for a large one takes as long as it did the first time.

If a connection stops working

Sift says on Home and in Settings when a source refuses its permission. It then reads that source no more, and it does not try again by itself. An empty list therefore never looks like good news.

Press Check now first. Sift renews the token and reads the source immediately. A permission that is still good works again at once. Connect the source again only if Check now gives you the same answer.

A permission can look dead while nothing is wrong with it. Sift runs in Docker, and the clock of a Docker virtual machine falls behind while your machine sleeps. Sift then keeps a token that the source already retired, and the source refuses it. Both of your sources can fail in the same minute in this way. Do not disconnect a source for this. A disconnection of Gmail removes every row of that mailbox, and the next read then takes as long as the first one did.

Ports

Postgres uses port 5433. The backend uses port 7777. The Vite dev server uses port 5174. These three ports avoid 5432, 5173 and the range 8080 to 8090. A local service stack often holds those. SIFT_PORT changes the port of the backend.

The dev server sends /api and /actuator to the backend. The API and the app then share one origin. The session cookie and the CSRF handshake behave exactly as they do in a built deployment.

Configuration

Everything is in .env. Copy that file from .env.example. The four SIFT_GITLAB_ values are necessary to connect GitLab. The three SIFT_GMAIL_ values are necessary to connect Gmail. The two sets are independent. Without a set, Sift tells you how to register that application.

Variable	Meaning
SIFT_ENCRYPTION_KEY	Base64 of 32 random bytes. Required. If you change it, Sift cannot decrypt any stored token.
SIFT_ALLOWED_EMAIL_DOMAINS	A comma separated list. If it is empty, any address can register. Use an empty value on a local instance only.
SIFT_GITLAB_URL	The address of your GitLab, for example https://gitlab.com.
SIFT_GITLAB_CLIENT_ID	The Application ID of your GitLab application.
SIFT_GITLAB_CLIENT_SECRET	The Secret of your GitLab application.
SIFT_GITLAB_REDIRECT_URI	The redirect URI of your GitLab application. It must agree with GitLab character for character.
SIFT_GMAIL_CLIENT_ID	The client ID of your Google OAuth client.
SIFT_GMAIL_CLIENT_SECRET	The client secret of your Google OAuth client.
SIFT_GMAIL_REDIRECT_URI	The redirect URI of your Google OAuth client. It must agree with Google character for character.
SIFT_SYNC_INTERVAL	The time between two reads of each source, as an ISO-8601 duration. The default is PT5M. Use a short value such as PT20S for a test. You then see a change in 20 seconds. Use PT2M while Sift reads a large mailbox for the first time.
POSTGRES_DB / POSTGRES_USER / POSTGRES_PASSWORD	The database credentials.

Check it works

The test suite needs Docker and nothing else.

cd backend && ./gradlew test

It runs against a real Postgres in a container, and it starts that container itself. It covers the parts where a silent error costs most: which items still wait for you, which items are complete, the search and the order of the list, how much of a mailbox one read takes, and the rule that you never see the items of another person.

verify/ holds the integration suites. They drive the real backend over HTTP against a stand-in GitLab or a stand-in Google. One of them drives the user interface in a browser.

verify/verify-participation.sh

Run one suite at a time. Each suite starts its own database and its own backend. Expect about one minute before the first check. verify/README.md explains what each suite covers.

GitHub Actions runs all of this on each push and on each pull request: the test suite and the frontend build in one workflow, and the integration suites and the browser suite in another.

Architecture

Sift is a backend-for-frontend. The browser holds a session cookie and nothing else. The source tokens stay on the server, and Sift encrypts them. Every call to a source goes through the server. The API and the app come from one origin, and there is no CORS configuration anywhere.

Sift also completes each approval on the server. No access token and no renewal token reaches your browser at any point.

Sift encrypts a token with AES-GCM before the token reaches the database. Nobody can read a token straight out of the database. On a machine only you can reach, this protects against two things: a casual look at the data, and an old backup. A hosted instance relies on the same protection.

Appearance

The dark theme is near-black. The light theme is a warm off-white. Brass marks the things that need you. The light theme is a design of its own. It is not an inversion of the dark theme. The type is Instrument Sans, with IBM Plex Mono for metadata. Both are self-hosted.

Each row has a left edge. The edge is brass while the row is unread. It turns grey after you read the row. The work that still waits for you is therefore the first thing you see down the page.

Each row also gives the reason it is in your list, next to the time. A colour groups the reasons: needs review, assigned to you, you were named, a discussion moved, something broke, mail arrived, or an outcome such as approved or merged. The reason is a word. You do not decode a colour.

The style of an action shows what the action costs you. Check a source, connect a source again and disconnect a source sit next to each other, and the three look different. The destructive one is red, and it asks you twice.

Pick one of three states: light, dark, or the setting of your system. Dark is the default. Sift remembers your choice between visits. The correct theme is in place before the first paint. The page never shows you the wrong theme first.