kaiyga/ygggo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

- Inter-node Protocol: Implemented a custom communication protocol based on Protobuf structures.

Browse Source 
- Bridge Node Support: Added functionality to manage mail (send/receive) on remote nodes via a bridge node. Full transparent management requires a private key on the bridge, while the remote node can operate with only a public key.
 - End-to-End Encryption (E2EE): Integrated E2EE using age. Encryption and decryption occur strictly at the bridge level; node administrators cannot access message content.
 - New Addressing Schema: Transitioned from nodepub@yggmail to userpub@nodepub.
 - Key Adaptation: Implemented ed25519 to x25519 coordinate adaptation, allowing signing keys to be used for encryption.

Changed
 - Address Encoding: Switched from hex16 to base32 for the address space.
 - Memory Optimization: Replaced direct memory allocations with streaming for mail processing to reduce RAM overhead.
 - Code Refactoring: Performed decomposition of the original inherited codebase to improve maintainability.

Fixed
 - Mail Updates: Fixed the mail retrieval mechanism to ensure near-instant updates for incoming messages.

Changes to be committed:
	modified:   .gitignore
	modified:   README.md
	modified:   cmd/yggmail/main.go
	modified:   go.mod
	modified:   go.sum
	new file:   internal/account/manager.go
	new file:   internal/account/session.go
	modified:   internal/imapserver/backend.go
	modified:   internal/imapserver/imap.go
	modified:   internal/imapserver/mailbox.go
	modified:   internal/imapserver/notify.go
	modified:   internal/imapserver/user.go
	new file:   internal/notify/notify.go
	new file:   internal/notify/remote/remote.go
	new file:   internal/remote/client.go
	new file:   internal/remote/handlers.go
	new file:   internal/remote/io.go
	new file:   internal/remote/server.go
	new file:   internal/remote/sign.go
	new file:   internal/remote/types/request.pb.go
	new file:   internal/remote/types/request.proto
	new file:   internal/remote/types/response.pb.go
	new file:   internal/remote/types/response.proto
	new file:   internal/shell/shell.go
	modified:   internal/smtpsender/sender.go
	modified:   internal/smtpserver/backend.go
	modified:   internal/smtpserver/session_local.go
	modified:   internal/smtpserver/session_remote.go
	modified:   internal/smtpserver/smtp.go
	new file:   internal/storage/filestore/filestore.go
	new file:   internal/storage/local/storage.go
	new file:   internal/storage/remote/mail.go
	new file:   internal/storage/remote/mailbox.go
	new file:   internal/storage/remote/queue.go
	new file:   internal/storage/remote/storage.go
	new file:   internal/storage/remote/watch.go
	modified:   internal/storage/sqlite3/sqlite3.go
	new file:   internal/storage/sqlite3/table_accounts.go
	modified:   internal/storage/sqlite3/table_mailboxes.go
	modified:   internal/storage/sqlite3/table_mails.go
	modified:   internal/storage/sqlite3/table_queue.go
	modified:   internal/storage/storage.go
	modified:   internal/storage/types/types.go
	modified:   internal/utils/address.go
	new file:   internal/utils/age/age.go
	new file:   internal/utils/age/bech32/bech32.go
	new file:   internal/utils/crypt.go
	new file:   internal/utils/e2ee/crypt.go
	new file:   internal/utils/e2ee/e2ee_test.go
	new file:   internal/utils/e2ee/msg.go
	new file:   internal/utils/x25519.go
	modified:   internal/welcome/welcome.go
This commit is contained in:
Kaiy Belist 2026-02-14 02:34:09 +05:00
parent 8bf3ba5f47
commit 39759db7c2
52 changed files with 6973 additions and 1069 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -1,2 +1,3 @@
*.db
.DS_Store
test

131
README.md
View File

@ -1,90 +1,103 @@
# Yggmail
# Ygggo
It's email, but not as you know it.
**Ygggo** is a fork of [Yggmail](https://www.google.com/search?q=https://github.com/yggdrasil-network/yggmail). It is a single-binary, all-in-one Mail Transfer Agent (MTA) that sends and receives email natively over the [Yggdrasil Network](https://yggdrasil-network.github.io/).
## Introduction
## 🚀 Key Improvements in Ygggo (Changelog)
Yggmail is a single-binary all-in-one mail transfer agent which sends and receives email natively over the [Yggdrasil Network](https://yggdrasil-network.github.io/).
This fork introduces several architectural shifts and performance optimizations:
* Yggmail runs just about anywhere you like — your inbox is stored right on your own machine;
* Implements IMAP and SMTP protocols for sending and receiving mail, so you can use your favourite client (hopefully);
* Mails are exchanged between Yggmail users using built-in Yggdrasil connectivity;
* All mail exchange traffic between any two Yggmail nodes is always end-to-end encrypted without exception;
* Yggdrasil and Yggmail nodes on the same network are discovered automatically using multicast or you can configure a static Yggdrasil peer.
* **New Addressing Scheme:** Migrated from `hex16` to **base32**. Addresses now follow the format: `<userpub>@<nodepub>`.
* **Inter-node Protocol:** Implemented a custom node-to-node communication protocol using **Protobuf** structures.
* **Bridge Architecture:** * Nodes can now act as bridges. You can connect to a remote node from a bridge to fetch or send mail transparently.
* Private keys are only required on the bridge; the end-node can operate with public keys only, simplifying administration.
Email addresses are based on your public key, like `89cd1ea25d99b8ccf29e454280313128c234ffb82aa0eb2e3496f6f156d063d0@yggmail`.
## Why?
* **End-to-End Encryption (E2EE):** * Integrated **age** encryption. Emails are encrypted at the source and decrypted only at the destination bridge.
* **Privacy:** Node administrators cannot see the content of the emails passing through or stored on the node.
There are all sorts of messaging services in the world but there is still a lot of value in asynchronous communication. Email is something that a lot of people understand reasonably well and there is still a huge volume of software in the world which supports email. Yggmail is designed to comply with the standards that people know and expect.
Yggdrasil is well-suited for ad-hoc mail delivery and allows Yggmail to work even in closed networks, where Internet or other connectivity is restricted or simply not available. It guarantees end-to-end encryption and handles networks with changing topologies reasonably well.
* **Mathematical Key Adaptation:** Leverages the fact that `ed25519` signing keys can be converted for `x25519` encryption, allowing your existing identity keys to handle E2EE seamlessly.
* **Performance & Refactoring:**
* **Streaming:** Switched from direct memory allocations to data streaming for mail handling.
* **Instant Sync:** Fixed mail retrieval updates; new mail is now detected and pushed almost instantly.
* **Code Quality:** Significant decomposition of the original codebase for better maintainability.
---
## Features
* **Sovereign Hosting:** Your inbox is stored on your own machine or node.
* **Standard Protocols:** Implements **IMAP** and **SMTP**, making it compatible with clients like Thunderbird, Outlook, or DeltaChat.
* **Automatic Discovery:** Discovers peers on the same LAN via multicast or connects to global static peers.
* **Native Privacy:** All traffic between nodes is end-to-end encrypted by Yggdrasil, with an additional layer of `age` encryption for the mail content.
---
## Quickstart
Use a recent version of Go to install Yggmail:
### 1. Installation
```
go install github.com/neilalexander/yggmail/cmd/yggmail@latest
```
Install the binary using Go:
It will then be installed into your `GOPATH`, so add that to your environment:
```
```bash
go install github.com/kaiyga/ygggo/cmd/yggmail@latest
export PATH=$PATH:`go env GOPATH`/bin
```
Create a mailbox and set your password. Your Yggmail database will automatically be created in your working directory if it doesn't already exist:
```
### 2. Configuration
Create your mailbox and set a password for your mail client:
```bash
yggmail -password
```
Start Yggmail, using the database in your working directory, with either multicast enabled, an [Yggdrasil static peer](https://publicpeers.neilalexander.dev/) specified or both:
```
yggmail -multicast
### 3. Execution
Start the node with multicast enabled or connect to a [public peer](https://publicpeers.neilalexander.dev/):
```bash
# Using a specific peer
yggmail -peer=tls://...
yggmail -multicast -peer=tls://...
# Using multicast for local discovery
yggmail -multicast
```
Your mail address will be printed in the log at startup. You will also use this as your username when you log into SMTP/IMAP.
*Your email address will be printed in the logs at startup.*
Connect your mail client to Yggmail. In the above example:
### 4. Client Setup
* SMTP is listening on `localhost` port 1025, username is your mail address, plain password authentication, no SSL/TLS
* IMAP is listening on `localhost` port 1143, username is your mail address, plain password authentication, no SSL/TLS
Connect your favorite mail client to the following local endpoints:
Then try sending a mail to another Yggmail user!
* **SMTP:** `localhost:1025` (No SSL/TLS, Plain Password)
* **IMAP:** `localhost:1143` (No SSL/TLS, Plain Password)
* **Username:** Your full Ygggo address.
---
## Parameters
The following command line switches are supported by the `yggmail` binary:
| Switch | Description |
| --- | --- |
| `-peer=...` | Connect to a specific Yggdrasil node (e.g., `tls://...`). |
| `-multicast` | Enable multicast discovery for LAN nodes. |
| `-database` | Path to the `yggmail.db` file (default is current directory). |
| `-smtp` | Listening address/port for SMTP (default `127.0.0.1:1025`). |
| `-imap` | Listening address/port for IMAP (default `127.0.0.1:1143`). |
| `-password` | Set your IMAP/SMTP password interactively. |
| `-passwordhash` | Set the IMAP/SMTP password using a pre-generated bcrypt hash. |
* `-peer=tls://...` or `-peer=tcp://...` — connect to a specific Yggdrasil node, like one of the [Public Peers](https://publicpeers.neilalexander.dev/);
* `-multicast` - enable multicast peer discovery for Yggdrasil nodes on your LAN
* `-mcastregexp=".*"` - regexp used in muticast peer discovery for interface name selection.
* `-database=/path/to/yggmail.db` — use a specific database file;
* `-smtp=listenaddr:port` — listen for SMTP on a specific address/port
* `-imap=listenaddr:port` — listen for IMAP on a specific address/port;
* `-password` — set your IMAP/SMTP password (doesn't matter if Yggmail is running or not, just make sure that Yggmail is pointing at the right database file or that you are in the right working directory).
* `-passwordhash` — Like `-password` however this sets what must be directly in the database. This assumes you passed a bcrypt hash
---
## Notes
## Important Notes
There are a few important notes:
* Yggmail needs to be running in order to receive inbound emails — it's therefore important to run Yggmail somewhere that will have good uptime;
* Yggmail tries to guarantee that senders are who they say they are. Your `From` address must be your Yggmail address;
* You can only email other Yggmail users, not regular email addresses on the public Internet;
* You may need to configure your client to allow "insecure" or "plaintext" authentication to IMAP/SMTP — this is because we don't support SSL/TLS on the IMAP/SMTP listeners yet;
* Yggmail won't transport mails larger than 1MB right now.
## Bugs
There are probably all sorts of bugs, but the ones that we know of are:
* IMAP behaviour might not be entirely spec-compliant in all cases, so your mileage with mail clients might vary;
* IMAP search isn't implemented yet and will instead return all mails.
The code's also a bit of a mess, so sorry about that.
* **Uptime:** Ygggo must be running to receive inbound emails.
* **Closed Ecosystem:** You can only email other Ygggo/Yggmail users. It does not route to the public Internet (Gmail, etc.).
* **Client Compatibility:** You may need to enable "Allow insecure authentication" in your mail client since the local listeners do not use SSL/TLS yet.
* **Size Limit:** Emails are currently capped at default **10mb** max **4GB**.

284
cmd/yggmail/main.go
View File

@ -1,5 +1,6 @@
/*
* Copyright (c) 2021 Neil Alexander
* Original Copyright (c) 2026 Kaiy Ragur
* Original Copyright (c) 2021 Neil Alexander
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -9,9 +10,7 @@
package main
import (
"bytes"
"crypto/ed25519"
"encoding/hex"
"flag"
"fmt"
"log"
@ -23,32 +22,41 @@ import (
"github.com/emersion/go-sasl"
"github.com/emersion/go-smtp"
"github.com/fatih/color"
"golang.org/x/term"
"net/http"
_ "net/http/pprof"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/config"
"github.com/neilalexander/yggmail/internal/imapserver"
"github.com/neilalexander/yggmail/internal/notify"
"github.com/neilalexander/yggmail/internal/remote"
"github.com/neilalexander/yggmail/internal/shell"
"github.com/neilalexander/yggmail/internal/smtpsender"
"github.com/neilalexander/yggmail/internal/smtpserver"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/filestore"
"github.com/neilalexander/yggmail/internal/storage/local"
"github.com/neilalexander/yggmail/internal/storage/sqlite3"
"github.com/neilalexander/yggmail/internal/transport"
"github.com/neilalexander/yggmail/internal/utils"
"github.com/neilalexander/yggmail/internal/welcome"
"golang.org/x/crypto/bcrypt"
)
type peerAddrList []string
func (i *peerAddrList) String() string {
return strings.Join(*i, ", ")
}
func (i *peerAddrList) String() string { return strings.Join(*i, ", ") }
func (i *peerAddrList) Set(value string) error {
*i = append(*i, value)
return nil
}
func main() {
go func() {
http.ListenAndServe("localhost:6060", nil)
}()
rawlog := log.New(color.Output, "", 0)
green := color.New(color.FgGreen).SprintfFunc()
log := log.New(rawlog.Writer(), fmt.Sprintf("[ %s ] ", green("Yggmail")), log.LstdFlags|log.Lmsgprefix)
@ -59,197 +67,161 @@ func main() {
imapaddr := flag.String("imap", "localhost:1143", "IMAP listen address")
multicast := flag.Bool("multicast", false, "Connect to Yggdrasil peers on your LAN")
mcastregexp := flag.String("mcastregexp", ".*", "Regexp for multicast")
password := flag.Bool("password", false, "Set a new IMAP/SMTP password")
passwordhash := flag.String("passwordhash", "", "Set a new IMAP/SMTP password (hash)")
flag.Var(&peerAddrs, "peer", "Connect to a specific Yggdrasil static peer (this option can be given more than once)")
shellMode := flag.String("sh", "", "-sh \"command args\"")
flag.Var(&peerAddrs, "peer", "Connect to a specific Yggdrasil static peer")
flag.Parse()
if flag.NFlag() == 0 {
fmt.Println("Yggmail must be started with either one or more Yggdrasil peers")
fmt.Println("specified, multicast enabled, or both.")
fmt.Println()
fmt.Println("Available options:")
fmt.Println()
flag.PrintDefaults()
os.Exit(0)
}
storage, err := sqlite3.NewSQLite3StorageStorage(*database)
db, err := sqlite3.NewSQLite3StorageStorage(*database)
if err != nil {
panic(err)
log.Fatalf("Failed to open database: %v", err)
}
defer storage.Close() // nolint:errcheck
log.Printf("Using database file %q\n", *database)
defer db.Close()
skStr, err := storage.ConfigGet("private_key")
fs, err := filestore.NewFileStore("./mails/")
if err != nil {
panic(err)
log.Fatalf("Failed to open filesystem: %v", err)
}
sk := make(ed25519.PrivateKey, ed25519.PrivateKeySize)
if skStr == "" {
if _, sk, err = ed25519.GenerateKey(nil); err != nil {
panic(err)
}
if err := storage.ConfigSet("private_key", hex.EncodeToString(sk)); err != nil {
panic(err)
}
log.Printf("Generated new server identity")
} else {
skBytes, err := hex.DecodeString(skStr)
if err != nil {
panic(err)
}
copy(sk, skBytes)
}
pk := sk.Public().(ed25519.PublicKey)
mailAddrUser := hex.EncodeToString(pk)
mailAddr := fmt.Sprintf("%s@%s", mailAddrUser, utils.Domain)
log.Printf("Mail address: %s\n", mailAddr)
storage := local.NewLocalStorage(db, fs, 0)
for _, name := range []string{"INBOX", "Outbox", "Sent"} {
if err := storage.MailboxCreate(name); err != nil {
panic(err)
}
}
welcome.Onboard(mailAddrUser, storage, log)
switch {
case password != nil && *password:
log.Println("Please enter your new password:")
password1, err := term.ReadPassword(int(os.Stdin.Fd()))
if err != nil {
panic(err)
}
fmt.Println()
log.Println("Please enter your new password again:")
password2, err := term.ReadPassword(int(os.Stdin.Fd()))
if err != nil {
panic(err)
}
fmt.Println()
if !bytes.Equal(password1, password2) {
log.Println("The supplied passwords do not match")
os.Exit(1)
}
// trim away whitespace of UTF-8 bytes now as string
finalPassword := strings.TrimSpace(string(password1))
// perform hash
hash, err := bcrypt.GenerateFromPassword([]byte(finalPassword), bcrypt.DefaultCost)
if err != nil {
log.Printf("bcrypt.GenerateFromPassword: %v\n", err)
os.Exit(1)
} else if err := storage.ConfigSetPassword(string(hash)); err != nil {
log.Println("Failed to set password:", err)
os.Exit(1)
}
log.Println("Password for IMAP and SMTP has been updated!")
os.Exit(0)
case passwordhash != nil && *passwordhash != "":
var hash = strings.TrimSpace(*passwordhash)
if len(hash) == 0 {
log.Println("Password hash cannot be blank")
os.Exit(1)
}
log.Printf("Using password hash: '%v'\n", hash)
if _, err := bcrypt.Cost(([]byte)(hash)); err != nil {
log.Printf("The provided hash is invalid %v\n", err)
os.Exit(1)
} else if err := storage.ConfigSetPassword(hash); err != nil {
log.Println("Failed to set password: ", err)
os.Exit(1)
}
log.Println("Password for IMAP and SMTP has been updated!")
case (multicast == nil || !*multicast) && len(peerAddrs) == 0:
log.Printf("You must specify either -peer, -multicast or both!")
os.Exit(0)
}
accManager := account.NewManager(storage, log)
nodeSk, nodePk := getID(storage, log)
cfg := &config.Config{
PublicKey: pk,
PrivateKey: sk,
PublicKey: nodePk,
PrivateKey: nodeSk,
}
transport, err := transport.NewYggdrasilTransport(rawlog, sk, pk, peerAddrs, *multicast, *mcastregexp)
if *shellMode != "" {
s := shell.NewShell(storage, log, cfg)
args := strings.Split(*shellMode, " ")
s.Run(args[0], args[1:])
return
}
checkFirstRun(accManager, storage, log, nodePk)
tr, err := transport.NewYggdrasilTransport(rawlog, nodeSk, nodePk, peerAddrs, *multicast, *mcastregexp)
if err != nil {
panic(err)
log.Fatalf("Failed to setup Yggdrasil: %v", err)
}
queues := smtpsender.NewQueues(cfg, log, transport, storage)
var notify *imapserver.IMAPNotify
// Notify bus
notify := notify.NewNotifyBus(log)
imapBackend := &imapserver.Backend{
Log: log,
Config: cfg,
Storage: storage,
}
// Imap Frontend
imapBackend := imapserver.NewBackend(
cfg,
log,
storage,
accManager,
tr,
notify,
)
_, notify, err = imapserver.NewIMAPServer(imapBackend, *imapaddr, true)
srv, err := imapserver.NewIMAPServer(imapBackend, *imapaddr, true)
if err != nil {
log.Fatal(err)
}
log.Println("Listening for IMAP on:", *imapaddr)
imapBackend.Server = srv
// Queues schedule
// For connect with Peers and Push Mail we sign request
nodeSigner := func(data []byte) ([]byte, error) {
return ed25519.Sign(nodeSk, data), nil
}
nodeRPCClient := remote.NewClient(
nodePk,
nodeSigner,
tr,
)
queues := smtpsender.NewQueues(cfg, log, tr, storage, nodeRPCClient, notify)
// SMTP-Frontend
go func() {
localBackend := &smtpserver.Backend{
Log: log,
Mode: smtpserver.BackendModeInternal,
Config: cfg,
// Config: cfg,
Storage: storage,
Queues: queues,
Notify: notify,
Account: accManager,
NodePublicKey: cfg.PublicKey,
}
localServer := smtp.NewServer(localBackend)
localServer.Addr = *smtpaddr
localServer.Domain = hex.EncodeToString(pk)
localServer.MaxMessageBytes = 1024 * 1024 * 32
localServer.MaxRecipients = 50
localServer.AllowInsecureAuth = true
localServer.EnableAuth(sasl.Login, func(conn *smtp.Conn) sasl.Server {
server := smtp.NewServer(localBackend)
server.Addr = *smtpaddr
server.Domain = utils.EncodeString(nodePk)
server.AllowInsecureAuth = true
server.EnableAuth(sasl.Login, func(conn *smtp.Conn) sasl.Server {
return sasl.NewLoginServer(func(username, password string) error {
_, err := localBackend.Login(nil, username, password)
return err
})
})
log.Println("Listening for SMTP on:", localServer.Addr)
if err := localServer.ListenAndServe(); err != nil {
log.Println("Listening for local SMTP on:", server.Addr)
if err := server.ListenAndServe(); err != nil {
log.Fatal(err)
}
}()
// RPC Mail-Server
remoteSrv := remote.NewRemoteServer(cfg.PublicKey, storage, accManager, log, notify, queues)
go func() {
overlayBackend := &smtpserver.Backend{
Log: log,
Mode: smtpserver.BackendModeExternal,
Config: cfg,
Storage: storage,
Queues: queues,
Notify: notify,
}
overlayServer := smtp.NewServer(overlayBackend)
overlayServer.Domain = hex.EncodeToString(pk)
overlayServer.MaxMessageBytes = 1024 * 1024 * 32
overlayServer.MaxRecipients = 50
overlayServer.AuthDisabled = true
if err := overlayServer.Serve(transport.Listener()); err != nil {
log.Fatal(err)
if err := remoteSrv.Serve(tr.Listener()); err != nil {
log.Fatalf("Remote Storage Server failed: %v", err)
}
}()
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
<-sigs
log.Println("Shutting down")
log.Println("Shutting down...")
}
func getID(s storage.Storage, l *log.Logger) (ed25519.PrivateKey, ed25519.PublicKey) {
skStr, _ := s.ConfigGet("private_key")
if skStr == "" {
_, sk, _ := ed25519.GenerateKey(nil)
s.ConfigSet("private_key", utils.EncodeString(sk))
l.Println("Generated new Yggdrasil Node Identity")
return sk, sk.Public().(ed25519.PublicKey)
}
skBytes, _ := utils.DecodeString(skStr)
sk := ed25519.PrivateKey(skBytes)
return sk, sk.Public().(ed25519.PublicKey)
}
func checkFirstRun(m *account.Manager, s storage.Storage, l *log.Logger, nodePk ed25519.PublicKey) {
accounts, _ := m.AccountList()
if len(accounts) > 0 {
return
}
l.Println("--- FIRST RUN: CREATE YOUR ACCOUNT ---")
pProvider := func() (string, error) {
p, err := account.DefaultProvider("Enter password for your new mail account: ")
if err != nil {
return "", err
}
p2, err := account.DefaultProvider("Comfirm your password: ")
if err != nil {
return "", err
}
if p == p2 {
return p2, nil
}
return "", fmt.Errorf("Password not match")
}
acc, err := m.CreateNewAccount(pProvider)
if err != nil {
l.Fatalf("Failed to create initial account: %v", err)
}
welcome.Onboard(acc.PublicKey, s, l)
l.Printf("SUCCESS! Your address is: %s@%s\n", acc.PublicKey, utils.EncodeString(nodePk))
l.Println("--------------------------------------")
}

4
go.mod
View File

@ -3,6 +3,7 @@ module github.com/neilalexander/yggmail
go 1.24.0
require (
filippo.io/edwards25519 v1.1.0
github.com/emersion/go-imap v1.2.1
github.com/emersion/go-imap-idle v0.0.0-20210907174914-db2568431445
github.com/emersion/go-imap-move v0.0.0-20210907172020-fe4558f9c872
@ -17,9 +18,12 @@ require (
go.uber.org/atomic v1.11.0
golang.org/x/crypto v0.45.0
golang.org/x/term v0.37.0
google.golang.org/protobuf v1.36.11
)
require (
filippo.io/age v1.3.1 // indirect
filippo.io/hpke v0.4.0 // indirect
github.com/Arceliar/ironwood v0.0.0-20251124020000-e1358f790504 // indirect
github.com/Arceliar/phony v0.0.0-20220903101357-530938a4b13d // indirect
github.com/bits-and-blooms/bitset v1.13.0 // indirect

10
go.sum
View File

@ -1,3 +1,9 @@
filippo.io/age v1.3.1 h1:hbzdQOJkuaMEpRCLSN1/C5DX74RPcNCk6oqhKMXmZi0=
filippo.io/age v1.3.1/go.mod h1:EZorDTYUxt836i3zdori5IJX/v2Lj6kWFU0cfh6C0D4=
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
filippo.io/hpke v0.4.0 h1:p575VVQ6ted4pL+it6M00V/f2qTZITO0zgmdKCkd5+A=
filippo.io/hpke v0.4.0/go.mod h1:EmAN849/P3qdeK+PCMkDpDm83vRHM5cDipBJ8xbQLVY=
github.com/Arceliar/ironwood v0.0.0-20251124020000-e1358f790504 h1:yQNDl2lGRV61efEug+cTnLYs7qnhnCm1TM42wZ+9xtE=
github.com/Arceliar/ironwood v0.0.0-20251124020000-e1358f790504/go.mod h1:SrrElc3FFMpYCODSr11jWbLFeOM8WsY+DbDY/l2AXF0=
github.com/Arceliar/phony v0.0.0-20220903101357-530938a4b13d h1:UK9fsWbWqwIQkMCz1CP+v5pGbsGoWAw6g4AyvMpm1EM=
@ -36,6 +42,8 @@ github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/gologme/log v1.3.0 h1:l781G4dE+pbigClDSDzSaaYKtiueHCILUa/qSDsmHAo=
github.com/gologme/log v1.3.0/go.mod h1:yKT+DvIPdDdDoPtqFrFxheooyVmoqi0BAsw+erN3wA4=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/hjson/hjson-go/v4 v4.5.0 h1:ZHLiZ+HaGqPOtEe8T6qY8QHnoEsAeBv8wqxniQAp+CY=
github.com/hjson/hjson-go/v4 v4.5.0/go.mod h1:4zx6c7Y0vWcm8IRyVoQJUHAPJLXLvbG6X8nk1RLigSo=
github.com/martinlindhe/base36 v1.0.0/go.mod h1:+AtEs8xrBpCeYgSLoY/aJ6Wf37jtBuR0s35750M27+8=
@ -115,5 +123,7 @@ golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

192
internal/account/manager.go Normal file
View File

@ -0,0 +1,192 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package account
import (
"crypto/ed25519"
"errors"
"fmt"
"log"
"os"
"strings"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/utils"
"golang.org/x/crypto/bcrypt"
"golang.org/x/term"
)
type Manager struct {
storage storage.Storage
log *log.Logger
}
type SecretProvider func() (string, error)
// Function for wrapping by your Provider
func DefaultProvider(msg string) (string, error) {
fmt.Print(msg)
p, err := term.ReadPassword(int(os.Stdin.Fd()))
password := strings.TrimSpace(string(p))
fmt.Println()
return password, err
}
func NewManager(storage storage.Storage, log *log.Logger) *Manager {
return &Manager{storage: storage, log: log}
}
func (m *Manager) AccountList() ([]*types.Account, error) { return m.storage.AccountList() }
func (m *Manager) CreateNewAccount(p SecretProvider) (*types.Account, error) {
_, sk, err := ed25519.GenerateKey(nil)
if err != nil {
return nil, fmt.Errorf("account.Manager.CreateNewAccount: GenerateKey: %w", err)
}
acc, err := m.CreateAccount(sk, p)
if err != nil {
return nil, fmt.Errorf("account.Manager.CreateNewAccount: %w", err)
}
return acc, nil
}
func (m *Manager) CreateAccount(sk ed25519.PrivateKey, p SecretProvider) (*types.Account, error) {
if len(sk) != ed25519.PrivateKeySize {
return nil, fmt.Errorf("account.Manager.CreateAccount: PrivateKeySize != 32")
}
password, err := p()
if err != nil {
return nil, fmt.Errorf("account.Manger.CreateAccount: SecretProvider: %w", err)
}
pk := sk.Public().(ed25519.PublicKey)
enSk, err := utils.AESEncrypt(utils.EncodeString(sk), password)
if err != nil {
return nil, fmt.Errorf("account.Manger.CreateAccount: encrypt privateKey: %w", err)
}
passwdhash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return nil, fmt.Errorf("account.Manager.CreateAccount: hash password: %w", err)
}
// nulling sec-data
sk = nil
password = ""
acc := &types.Account{
PublicKey: utils.EncodeString(pk),
// TODO Added Aliases.__.
Aliases: "",
PasswdHash: string(passwdhash),
Privatekey: enSk,
}
if err := m.storage.AccountCreate(acc); err != nil {
return nil, fmt.Errorf("account.Manager.CreateAccount: storage create account: %w", err)
}
for _, folder := range []string{"INBOX", "Outbox", "Sent", "Trash", "Drafts"} {
m.storage.MailboxCreate(acc.PublicKey, folder)
}
return acc, nil
}
func (m *Manager) AddAccount(pubkey string) (*types.Account, error) {
pkb, err := utils.DecodeString(pubkey)
if err != nil {
return nil, fmt.Errorf("account.Manager.AddAccount: can't decode account key")
}
if len(pkb) != ed25519.PublicKeySize {
return nil, fmt.Errorf("account.Manager.AddAccount: wrong key size")
}
acc := &types.Account{
PublicKey: utils.EncodeString(pkb),
// TODO Added Aliases.__.
Aliases: "",
PasswdHash: "",
Privatekey: "",
}
if err := m.storage.AccountCreate(acc); err != nil {
return nil, fmt.Errorf("account.Manager.CreateAccount: storage create account: %w", err)
}
for _, folder := range []string{"INBOX", "Outbox", "Sent", "Trash", "Drafts"} {
m.storage.MailboxCreate(acc.PublicKey, folder)
}
return acc, nil
}
func (m *Manager) GetAccount(pubkey string) (*types.Account, error) {
acc, err := m.storage.AccountGet(pubkey)
if err != nil {
return nil, fmt.Errorf("failed to find: %w", err)
}
if acc == nil {
return nil, fmt.Errorf("failed to find: wrong credentials : %w", err)
// return nil, fmt.Errorf("failed to find: wrong username : %w", err)
}
return acc, nil
}
func (m *Manager) DeleteAccount(pubkey string) error {
if err := m.storage.AccountDelete(pubkey); err != nil {
return fmt.Errorf("account.Manager.DeleteAccount: %w", err)
}
return nil
}
var ErrInvalidCredentials = errors.New("Invalid credentials")
func (m *Manager) Login(username, password string) (session *Session, err error) {
_, aPk, err := utils.ParseAddress(username)
if err != nil {
return nil, fmt.Errorf("failed to authenticate: wrong domain or username")
}
// Trim away whitespace of UTF-8 bytes now as string
password = strings.TrimSpace(password)
// For security all errors is same. Uncomment for debug!
acc, err := m.storage.AccountGet(utils.EncodeString(aPk))
if err != nil || acc == nil {
// return nil, fmt.Errorf("failed to authenticate: storage: %w", err)
return nil, ErrInvalidCredentials
}
if err := bcrypt.CompareHashAndPassword([]byte(acc.PasswdHash), []byte(password)); err != nil {
// return nil, fmt.Errorf("failed to authenticate: wrong password : %w", err)
return nil, ErrInvalidCredentials
}
// Autheticated!
if acc.PasswdHash == "" || acc.Privatekey == "" {
return nil, fmt.Errorf("failed have not account.PrivateKey for client mode!")
}
aSkStr, err := utils.AESDecrypt(acc.Privatekey, password)
if err != nil {
return nil, fmt.Errorf("failed to authenticate: Privkey AESDecrypt: %w", err)
}
password = ""
aSk, err := utils.DecodeString(aSkStr)
if err != nil {
return nil, fmt.Errorf("failed to authenticate: Privkey Parse: %w", err)
}
aSkStr = ""
return NewSession(acc, ed25519.PrivateKey(aSk)), nil
}
func (m *Manager) ChangePassword(pubkey string) {
panic("Not Implemented")
}

83
internal/account/session.go Normal file
View File

@ -0,0 +1,83 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package account
import (
"crypto/ed25519"
"fmt"
"io"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/utils"
"github.com/neilalexander/yggmail/internal/utils/age"
"golang.org/x/crypto/bcrypt"
)
// Incapsulate all logic with cryptography by account
type Session struct {
account *types.Account
privKey ed25519.PrivateKey
}
// Constructor
func NewSession(a *types.Account, p ed25519.PrivateKey) *Session {
return &Session{
account: a,
privKey: p,
}
}
func (s *Session) Username() string { return s.account.PublicKey }
func (s *Session) Publickey() ed25519.PublicKey { return s.privKey.Public().(ed25519.PublicKey) }
func (s *Session) GetAccount() *types.Account { return s.account }
func (s *Session) Sign(data []byte) (sign []byte, err error) {
sign = ed25519.Sign(s.privKey, data)
return sign, nil
}
func (s *Session) Decrypt(r io.Reader) (io.Reader, error) {
i, err := age.Identity(s.privKey)
if err != nil {
return nil, err
}
return age.DecryptArmor(r, i)
}
func (s *Session) Encrypt(w io.Writer, rcpt ...ed25519.PublicKey) (io.WriteCloser, error) {
r, err := age.Recipient(rcpt...)
if err != nil {
return nil, err
}
return age.EncryptArmor(w, r...)
}
func (s *Session) ChangePassword(p SecretProvider) (enSk string, passwdHash []byte, err error) {
sk := s.privKey
if len(sk) != ed25519.PrivateKeySize {
return "", nil, fmt.Errorf("session.ChangePassword:: PrivateKeySize != 32")
}
password, err := p()
if err != nil {
return "", nil, fmt.Errorf("session.ChangePassword: SecretProvider: %w", err)
}
enSk, err = utils.AESEncrypt(utils.EncodeString(sk), password)
if err != nil {
return "", nil, fmt.Errorf("session.ChangePassword: encrypt privateKey: %w", err)
}
passwdhash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return "", nil, fmt.Errorf("session.ChangePassword:: hash password: %w", err)
}
// nulling sec-data
password = ""
return enSk, passwdhash, nil
}

73
internal/imapserver/backend.go
View File

@ -9,14 +9,18 @@
package imapserver
import (
"encoding/hex"
"fmt"
"log"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/backend"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/config"
"github.com/neilalexander/yggmail/internal/notify"
nfremote "github.com/neilalexander/yggmail/internal/notify/remote"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/remote"
"github.com/neilalexander/yggmail/internal/transport"
"github.com/neilalexander/yggmail/internal/utils"
)
@ -25,28 +29,67 @@ type Backend struct {
Log *log.Logger
Storage storage.Storage
Server *IMAPServer
Account *account.Manager
Transport transport.Transport
Notify *notify.Notify
updates chan backend.Update
}
func NewBackend(
config *config.Config,
log *log.Logger,
storage storage.Storage,
// server *IMAPServer,
account *account.Manager,
transport transport.Transport,
notify *notify.Notify,
) *Backend {
b := &Backend{
Config: config,
Log: log,
Storage: storage,
//Server: server,
Account: account,
Transport: transport,
Notify: notify,
updates: make(chan backend.Update, 100),
}
b.listenUpdates()
return b
}
func (b *Backend) Updates() <-chan backend.Update {
b.Log.Println("Updates is loaded!")
return b.updates
}
func (b *Backend) Login(conn *imap.ConnInfo, username, password string) (backend.User, error) {
// If our username is email-like, then take just the localpart
if pk, err := utils.ParseAddress(username); err == nil {
if !pk.Equal(b.Config.PublicKey) {
b.Log.Println("Failed to authenticate IMAP user due to wrong domain", hex.EncodeToString(pk), hex.EncodeToString(b.Config.PublicKey))
return nil, fmt.Errorf("failed to authenticate: wrong domain in username")
dPk, _, err := utils.ParseAddress(username)
if err != nil {
return nil, fmt.Errorf("Failed ")
}
remoteDomain := !dPk.Equal(b.Config.PublicKey)
session, err := b.Account.Login(username, password)
if err != nil {
return nil, fmt.Errorf("IMAP: Error to authenticate: %w", err)
}
username = hex.EncodeToString(b.Config.PublicKey)
if authed, err := b.Storage.ConfigTryPassword(password); err != nil {
b.Log.Printf("Failed to authenticate IMAP user %q due to error: %s", username, err)
return nil, fmt.Errorf("failed to authenticate: %w", err)
} else if !authed {
b.Log.Printf("Failed to authenticate IMAP user %q\n", username)
return nil, backend.ErrInvalidCredentials
defer b.Log.Printf("IMAP: Authenticated user from %s as %q\n", conn.RemoteAddr.String(), username)
var storage storage.Storage
if remoteDomain {
storage = remote.NewRemoteStorage(dPk, session, b.Transport)
notifer := nfremote.NewRemoteNotifer(dPk, session.Publickey(), session.Sign, b.Transport)
notifer.StartRemoteNotifer(b.Notify, session.Publickey(), "INBOX")
} else {
storage = b.Storage
}
defer b.Log.Printf("Authenticated IMAP user from %s as %q\n", conn.RemoteAddr.String(), username)
user := &User{
backend: b,
username: username,
storage: storage,
username: session.Username(),
session: session,
conn: conn,
log: b.Log,
}

46
internal/imapserver/imap.go
View File

@ -10,6 +10,8 @@ package imapserver
import (
"log"
"net"
"time"
idle "github.com/emersion/go-imap-idle"
move "github.com/emersion/go-imap-move"
@ -20,31 +22,61 @@ import (
type IMAPServer struct {
server *server.Server
backend *Backend
notify *IMAPNotify
}
func NewIMAPServer(backend *Backend, addr string, insecure bool) (*IMAPServer, *IMAPNotify, error) {
type tcpNoDelayListener struct {
net.Listener
}
func (l *tcpNoDelayListener) Accept() (net.Conn, error) {
conn, err := l.Listener.Accept()
if err != nil {
return nil, err
}
if tcpConn, ok := conn.(*net.TCPConn); ok {
tcpConn.SetNoDelay(true)
tcpConn.SetKeepAlive(true)
tcpConn.SetKeepAlivePeriod(time.Second * 5)
}
return conn, nil
}
func NewIMAPServer(backend *Backend, addr string, insecure bool) (*IMAPServer, error) {
s := &IMAPServer{
server: server.New(backend),
backend: backend,
}
s.notify = NewIMAPNotify(s.server, backend.Log)
s.server.Addr = addr
s.server.AllowInsecureAuth = insecure
//s.server.Debug = os.Stdout
// s.server.Debug = func() io.Writer {
// return log.New(os.Stdout, "IMAP :", log.Default().Flags()).Writer()
// }()
s.server.Enable(idle.NewExtension())
s.server.Enable(move.NewExtension())
// s.server.Enable(s.notify)
s.server.EnableAuth(sasl.Login, func(conn server.Conn) sasl.Server {
return sasl.NewLoginServer(func(username, password string) error {
_, err := s.backend.Login(nil, username, password)
return err
})
})
// go func() {
// if err := s.server.ListenAndServe(); err != nil {
// log.Fatal(err)
// }
// }()
go func() {
if err := s.server.ListenAndServe(); err != nil {
l, err := net.Listen("tcp", s.server.Addr)
if err != nil {
log.Fatal(err)
}
// Оборачиваем листенер в наш тюнингованный вариант
if err := s.server.Serve(&tcpNoDelayListener{l}); err != nil {
log.Fatal(err)
}
}()
return s, s.notify, nil
return s, nil
}

561
internal/imapserver/mailbox.go
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2021 Neil Alexander
* (Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -10,40 +10,301 @@ package imapserver
import (
"bufio"
"bytes"
"fmt"
"io"
"time"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/backend/backendutil"
"github.com/emersion/go-message/textproto"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/utils/e2ee"
)
type Mailbox struct {
backend *Backend
storage storage.Storage
name string
user *User
}
func (mbox *Mailbox) Name() string {
return mbox.name
}
func (mbox *Mailbox) Info() (*imap.MailboxInfo, error) {
return &imap.MailboxInfo{
Attributes: []string{},
Delimiter: "/",
Name: mbox.name,
}, nil
}
func (mbox *Mailbox) Status(items []imap.StatusItem) (*imap.MailboxStatus, error) {
pk := mbox.user.username
status := imap.NewMailboxStatus(mbox.name, items)
status.PermanentFlags = []string{"\\Seen", "\\Answered", "\\Flagged", "\\Deleted"}
status.Flags = status.PermanentFlags
for _, name := range items {
switch name {
case imap.StatusMessages:
count, _ := mbox.storage.MailCount(pk, mbox.name)
status.Messages = uint32(count)
case imap.StatusUidNext:
id, _ := mbox.storage.MailNextID(pk, mbox.name)
status.UidNext = uint32(id)
case imap.StatusUidValidity:
status.UidValidity = 1
case imap.StatusUnseen:
unseen, _ := mbox.storage.MailUnseen(pk, mbox.name)
status.Unseen = uint32(unseen)
}
}
return status, nil
}
func (mbox *Mailbox) ListMessages(uid bool, seqSet *imap.SeqSet, items []imap.FetchItem, ch chan<- *imap.Message) error {
defer close(ch)
pk := mbox.user.username
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return err
}
for _, id := range ids {
mseq, m, err := mbox.storage.MailSelect(pk, mbox.name, int(id))
if err != nil {
continue
}
// n, _ := io.ReadAll(m.Body.(io.ReadSeeker))
// mbox.user.log.Printf("Body preview: \n\n %q \n\n", string(n))
// m.Body.(io.ReadSeeker).Seek(0, io.SeekStart)
m.Body, err = e2ee.DecryptFilter(m.Body.(io.ReadSeekCloser), mbox.user.session.Decrypt)
if err != nil {
mbox.user.log.Println("decrypter error: ", err)
return err
}
msg, err := mbox.fetchMessage(mseq, m, items)
if err != nil {
continue
}
ch <- msg
}
return nil
}
func (mbox *Mailbox) fetchMessage(seqNum int, m *types.Mail, items []imap.FetchItem) (*imap.Message, error) {
fetched := imap.NewMessage(uint32(m.ID), items)
fetched.SeqNum = uint32(seqNum)
fetched.Uid = uint32(m.ID)
if !mbox.needsBody(items) || m.Body == nil {
return mbox.fillMetadataOnly(fetched, m, items), nil
}
defer m.Body.Close()
br := bufio.NewReader(m.Body)
hdr, hdrErr := textproto.ReadHeader(br)
if hdrErr != nil {
return mbox.fillMetadataOnly(fetched, m, items), nil
}
for _, item := range items {
switch item {
case imap.FetchEnvelope:
fetched.Envelope, _ = backendutil.FetchEnvelope(hdr)
case imap.FetchBody, imap.FetchBodyStructure:
fetched.BodyStructure, _ = backendutil.FetchBodyStructure(hdr, br, item == imap.FetchBodyStructure)
case imap.FetchRFC822Size:
fetched.Size = uint32(m.Size)
case imap.FetchInternalDate:
fetched.InternalDate = m.Date
case imap.FetchFlags:
fetched.Flags = mbox.formatFlags(m)
default:
if section, err := imap.ParseBodySectionName(item); err == nil {
l, _ := backendutil.FetchBodySection(hdr, br, section)
fetched.Body[section] = l
}
}
}
return fetched, nil
}
func (mbox *Mailbox) needsBody(items []imap.FetchItem) bool {
for _, item := range items {
switch item {
case imap.FetchEnvelope, imap.FetchBody, imap.FetchBodyStructure:
return true
default:
if _, err := imap.ParseBodySectionName(item); err == nil {
return true
}
}
}
return false
}
func (mbox *Mailbox) fillMetadataOnly(dest *imap.Message, m *types.Mail, items []imap.FetchItem) *imap.Message {
for _, item := range items {
switch item {
case imap.FetchFlags:
dest.Flags = mbox.formatFlags(m)
case imap.FetchInternalDate:
dest.InternalDate = m.Date
case imap.FetchRFC822Size:
dest.Size = uint32(m.Size)
}
}
return dest
}
func (mbox *Mailbox) CreateMessage(flags []string, date time.Time, body imap.Literal) error {
id, err := mbox.storage.MailCreate(mbox.user.username, mbox.name, body, body.Len())
if err != nil {
return err
}
seen, answered, flagged, deleted := mbox.parseFlags(flags)
return mbox.storage.MailUpdateFlags(mbox.user.username, mbox.name, id, seen, answered, flagged, deleted)
}
func (mbox *Mailbox) CopyMessages(uid bool, seqSet *imap.SeqSet, destName string) error {
pk := mbox.user.username
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return err
}
for _, id := range ids {
_, mail, err := mbox.storage.MailSelect(pk, mbox.name, int(id))
if err != nil {
continue
}
newID, err := mbox.storage.MailCreate(pk, destName, mail.Body, int(mail.Size))
mail.Body.Close()
if err == nil {
_ = mbox.storage.MailUpdateFlags(pk, destName, newID, mail.Seen, mail.Answered, mail.Flagged, mail.Deleted)
}
}
return nil
}
func (mbox *Mailbox) UpdateMessagesFlags(uid bool, seqSet *imap.SeqSet, op imap.FlagsOp, flags []string) error {
pk := mbox.user.username
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return err
}
for _, id := range ids {
_, mail, err := mbox.storage.MailSelect(pk, mbox.name, int(id))
if err != nil {
continue
}
mail.Body.Close() // Close body. Read Flags
seen, ans, flg, del := mbox.calculateNewFlags(mail, op, flags)
_ = mbox.storage.MailUpdateFlags(pk, mbox.name, int(id), seen, ans, flg, del)
}
return nil
}
// calculateNewFlags выносит логику битовых (флаговых) операций из общего цикла
func (mbox *Mailbox) calculateNewFlags(mail *types.Mail, op imap.FlagsOp, flags []string) (s, a, f, d bool) {
s, a, f, d = mail.Seen, mail.Answered, mail.Flagged, mail.Deleted
ns, na, nf, nd := mbox.parseFlags(flags)
switch op {
case imap.SetFlags:
s, a, f, d = ns, na, nf, nd
case imap.AddFlags:
if ns {
s = true
}
if na {
a = true
}
if nf {
f = true
}
if nd {
d = true
}
case imap.RemoveFlags:
if ns {
s = false
}
if na {
a = false
}
if nf {
f = false
}
if nd {
d = false
}
}
return
}
func (mbox *Mailbox) parseFlags(flags []string) (s, a, f, d bool) {
for _, flag := range flags {
switch flag {
case "\\Seen":
s = true
case "\\Answered":
a = true
case "\\Flagged":
f = true
case "\\Deleted":
d = true
}
}
return
}
func (mbox *Mailbox) formatFlags(mail *types.Mail) []string {
f := []string{}
if mail.Seen {
f = append(f, "\\Seen")
}
if mail.Answered {
f = append(f, "\\Answered")
}
if mail.Flagged {
f = append(f, "\\Flagged")
}
if mail.Deleted {
f = append(f, "\\Deleted")
}
return f
}
func (mbox *Mailbox) getIDsFromSeqSet(uid bool, seqSet *imap.SeqSet) ([]int32, error) {
var ids []int32
pk := mbox.user.username
for _, set := range seqSet.Set {
if set.Stop == 0 {
next, err := mbox.backend.Storage.MailNextID(mbox.name)
if err != nil {
return nil, fmt.Errorf("mbox.backend.Storage.MailNextID: %w", err)
stop := set.Stop
if stop == 0 {
// In IMAP 0 = All
next, _ := mbox.storage.MailNextID(pk, mbox.name)
stop = uint32(next - 1)
}
set.Stop = uint32(next - 1)
}
for i := set.Start; i <= set.Stop; i++ {
for i := set.Start; i <= stop; i++ {
if !uid {
pid, err := mbox.backend.Storage.MailIDForSeq(mbox.name, int(i))
if err != nil {
return nil, fmt.Errorf("mbox.backend.Storage.MailIDForSeq: %w", err)
}
// Convert uid by db
pid, err := mbox.storage.MailIDForSeq(pk, mbox.name, int(i))
if err == nil {
ids = append(ids, int32(pid))
}
} else {
ids = append(ids, int32(i))
}
@ -52,283 +313,31 @@ func (mbox *Mailbox) getIDsFromSeqSet(uid bool, seqSet *imap.SeqSet) ([]int32, e
return ids, nil
}
func (mbox *Mailbox) Name() string {
return mbox.name
}
func (mbox *Mailbox) Info() (*imap.MailboxInfo, error) {
info := &imap.MailboxInfo{
Attributes: []string{},
Delimiter: "/",
Name: mbox.name,
}
return info, nil
}
func (mbox *Mailbox) Status(items []imap.StatusItem) (*imap.MailboxStatus, error) {
status := imap.NewMailboxStatus(mbox.name, items)
status.PermanentFlags = []string{
"\\Seen", "\\Answered", "\\Flagged", "\\Deleted",
}
status.Flags = status.PermanentFlags
for _, name := range items {
switch name {
case imap.StatusMessages:
count, err := mbox.backend.Storage.MailCount(mbox.name)
if err != nil {
return nil, fmt.Errorf("mbox.backend.Storage.MailCount: %w", err)
}
status.Messages = uint32(count)
case imap.StatusUidNext:
id, err := mbox.backend.Storage.MailNextID(mbox.name)
if err != nil {
return nil, fmt.Errorf("mbox.backend.Storage.MailNextID: %w", err)
}
status.UidNext = uint32(id)
case imap.StatusUidValidity:
status.UidValidity = 1
case imap.StatusRecent:
status.Recent = 0 // TODO
case imap.StatusUnseen:
unseen, err := mbox.backend.Storage.MailUnseen(mbox.name)
if err != nil {
return nil, fmt.Errorf("mbox.backend.Storage.MailUnseen: %w", err)
}
status.Unseen = uint32(unseen)
}
}
return status, nil
}
func (mbox *Mailbox) SetSubscribed(subscribed bool) error {
return mbox.backend.Storage.MailboxSubscribe(mbox.name, subscribed)
return mbox.storage.MailboxSubscribe(mbox.user.username, mbox.name, subscribed)
}
func (mbox *Mailbox) Check() error {
return nil
}
func (mbox *Mailbox) ListMessages(uid bool, seqSet *imap.SeqSet, items []imap.FetchItem, ch chan<- *imap.Message) error {
defer close(ch)
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return fmt.Errorf("mbox.getIDsFromSeqSet: %w", err)
}
for _, id := range ids {
mseq, mail, err := mbox.backend.Storage.MailSelect(mbox.name, int(id))
if err != nil {
continue
}
fetched := imap.NewMessage(uint32(id), items)
fetched.SeqNum = uint32(mseq)
fetched.Uid = uint32(mail.ID)
get := func() (io.Reader, textproto.Header, error) {
bodyreader := bufio.NewReader(bytes.NewReader(mail.Mail))
hdr, err := textproto.ReadHeader(bodyreader)
if err != nil {
return nil, textproto.Header{}, fmt.Errorf("textproto.ReadHeader: %w", err)
}
return bodyreader, hdr, err
}
for _, item := range items {
switch item {
case imap.FetchEnvelope:
_, hdr, err := get()
if err != nil {
continue
}
if fetched.Envelope, err = backendutil.FetchEnvelope(hdr); err != nil {
continue
}
case imap.FetchBody, imap.FetchBodyStructure:
bodyreader, hdr, err := get()
if err != nil {
continue
}
if fetched.BodyStructure, err = backendutil.FetchBodyStructure(hdr, bodyreader, item == imap.FetchBodyStructure); err != nil {
continue
}
case imap.FetchFlags:
fetched.Flags = []string{}
if mail.Seen {
fetched.Flags = append(fetched.Flags, "\\Seen")
}
if mail.Answered {
fetched.Flags = append(fetched.Flags, "\\Answered")
}
if mail.Flagged {
fetched.Flags = append(fetched.Flags, "\\Flagged")
}
if mail.Deleted {
fetched.Flags = append(fetched.Flags, "\\Deleted")
}
case imap.FetchInternalDate:
fetched.InternalDate = mail.Date
case imap.FetchRFC822Size:
fetched.Size = uint32(len(mail.Mail))
case imap.FetchUid:
fetched.Uid = uint32(id)
default:
section, err := imap.ParseBodySectionName(item)
if err != nil {
continue
}
bodyreader, hdr, err := get()
if err != nil {
continue
}
l, err := backendutil.FetchBodySection(hdr, bodyreader, section)
if err != nil {
continue
}
fetched.Body[section] = l
}
}
ch <- fetched
}
return nil
}
func (mbox *Mailbox) Check() error { return nil }
func (mbox *Mailbox) SearchMessages(uid bool, criteria *imap.SearchCriteria) ([]uint32, error) {
return mbox.backend.Storage.MailSearch(mbox.name)
}
func (mbox *Mailbox) CreateMessage(flags []string, date time.Time, body imap.Literal) error {
b, err := io.ReadAll(body)
if err != nil {
return fmt.Errorf("b.ReadFrom: %w", err)
}
id, err := mbox.backend.Storage.MailCreate(mbox.name, b)
if err != nil {
return fmt.Errorf("mbox.backend.Storage.MailCreate: %w", err)
}
for _, flag := range flags {
var seen, answered, flagged, deleted bool
switch flag {
case "\\Seen":
seen = true
case "\\Answered":
answered = true
case "\\Flagged":
flagged = true
case "\\Deleted":
deleted = true
}
if err := mbox.backend.Storage.MailUpdateFlags(
mbox.name, id, seen, answered, flagged, deleted,
); err != nil {
return err
}
}
return nil
}
func (mbox *Mailbox) UpdateMessagesFlags(uid bool, seqSet *imap.SeqSet, op imap.FlagsOp, flags []string) error {
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return fmt.Errorf("mbox.getIDsFromSeqSet: %w", err)
}
for _, id := range ids {
var mail *types.Mail
if op != imap.SetFlags {
var err error
_, mail, err = mbox.backend.Storage.MailSelect(mbox.name, int(id))
if err != nil {
return fmt.Errorf("mbox.backend.Storage.MailSelect: %w", err)
}
}
for _, flag := range flags {
switch flag {
case "\\Seen":
mail.Seen = op != imap.RemoveFlags
case "\\Answered":
mail.Answered = op != imap.RemoveFlags
case "\\Flagged":
mail.Flagged = op != imap.RemoveFlags
case "\\Deleted":
mail.Deleted = op != imap.RemoveFlags
}
}
if err := mbox.backend.Storage.MailUpdateFlags(
mbox.name, int(mail.ID), mail.Seen,
mail.Answered, mail.Flagged, mail.Deleted,
); err != nil {
return err
}
}
return nil
}
func (mbox *Mailbox) CopyMessages(uid bool, seqSet *imap.SeqSet, destName string) error {
if destName == "Outbox" {
return fmt.Errorf("can't copy into Outbox as it is a protected folder")
}
ids, err := mbox.getIDsFromSeqSet(uid, seqSet)
if err != nil {
return fmt.Errorf("mbox.getIDsFromSeqSet: %w", err)
}
for _, id := range ids {
_, mail, err := mbox.backend.Storage.MailSelect(mbox.name, int(id))
if err != nil {
return fmt.Errorf("mbox.backend.Storage.MailSelect: %w", err)
}
pid, err := mbox.backend.Storage.MailCreate(destName, mail.Mail)
if err != nil {
return fmt.Errorf("mbox.backend.Storage.MailCreate: %w", err)
}
if err = mbox.backend.Storage.MailUpdateFlags(
mbox.name, pid, mail.Seen, mail.Answered, mail.Flagged, mail.Deleted,
); err != nil {
return fmt.Errorf("mbox.backend.Storage.MailUpdateFlags: %w", err)
}
}
return nil
return mbox.storage.MailSearch(mbox.user.username, mbox.name)
}
func (mbox *Mailbox) Expunge() error {
return mbox.backend.Storage.MailExpunge(mbox.name)
return mbox.storage.MailExpunge(mbox.user.username, mbox.name)
}
func (mbox *Mailbox) MoveMessages(uid bool, seqset *imap.SeqSet, dest string) error {
if dest == "Outbox" {
return fmt.Errorf("can't copy into Outbox as it is a protected folder")
}
pk := mbox.user.username
ids, err := mbox.getIDsFromSeqSet(uid, seqset)
if err != nil {
return fmt.Errorf("mbox.getIDsFromSeqSet: %w", err)
return err
}
for _, id := range ids {
if err := mbox.backend.Storage.MailMove(mbox.name, int(id), dest); err != nil {
return err
}
if nid, err := mbox.storage.MailMove(pk, mbox.name, int(id), dest); err == nil {
// Delete mail from Outbox before send in Queue
if mbox.name == "Outbox" {
if err := mbox.backend.Storage.QueueDeleteDestinationForID("Outbox", int(id)); err != nil {
return err
_ = mbox.storage.QueueDeleteDestinationForID(mbox.user.Username(), "Outbox", int(nid))
}
}
}

84
internal/imapserver/notify.go
View File

@ -9,65 +9,41 @@
package imapserver
import (
"fmt"
"log"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/server"
"github.com/emersion/go-imap/backend"
"github.com/neilalexander/yggmail/internal/notify"
)
type IMAPNotifyHandler struct {
imap.Command
}
// github.com/emersion/go-imap/backend/updates.go
func (b *Backend) listenUpdates() {
allEvents := b.Notify.Subscribe(notify.AllUsers)
func (h *IMAPNotifyHandler) Handle(conn server.Conn) error {
// TODO: Support setting NOTIFY subscriptions or not
return nil
}
type IMAPNotify struct {
server *server.Server
log *log.Logger
}
func (ext *IMAPNotify) Capabilities(c server.Conn) []string {
if c.Context().State&imap.AuthenticatedState != 0 {
return []string{"NOTIFY"}
go func() {
for event := range allEvents {
if ev, ok := event.(*notify.EmailEvent); ok {
// b.updates <- &backend.MailboxUpdate{
// Update: backend.NewUpdate(ev.GetAccountPK(), ev.Mailbox()),
// MailboxStatus: &imap.MailboxStatus{
// Name: ev.MailBox,
// Messages: uint32(ev.Count),
// UidNext: uint32(ev.MessageId),
// },
// }
// b.updates <- &backend.MessageUpdate{
// Update: backend.NewUpdate(ev.AccountPK, ev.MailBox),
// Message: imap.NewMessage(uint32(ev.Count), []imap.FetchItem{imap.FetchAll}),
// }
b.updates <- &backend.MailboxUpdate{
Update: backend.NewUpdate(ev.GetAccountPK(), ev.MailBox),
MailboxStatus: &imap.MailboxStatus{
Name: ev.MailBox,
Messages: uint32(ev.Count),
UidNext: uint32(ev.MessageId),
Items: map[imap.StatusItem]interface{}{imap.StatusMessages: "", imap.StatusUidNext: ev.Count},
},
}
return nil
}
func (ext *IMAPNotify) Command(name string) server.HandlerFactory {
if name != "NOTIFY" {
return nil
b.Log.Println("IMAP Notify: ", ev.Username())
}
return func() server.Handler {
return &IMAPNotifyHandler{}
}
}
func (ext *IMAPNotify) NotifyNew(id, count int) error {
ext.server.ForEachConn(func(c server.Conn) {
var resptype imap.StatusRespType
if mailbox := c.Context().Mailbox; mailbox != nil && mailbox.Name() == "INBOX" {
resptype = imap.StatusRespType(
fmt.Sprintf("EXISTS %d", id),
)
} else {
resptype = imap.StatusRespType(
fmt.Sprintf("STATUS INBOX (UIDNEXT %d MESSAGES %d)", id+1, count),
)
}
_ = c.WriteResp(&imap.StatusResp{
Type: resptype,
})
})
return nil
}
func NewIMAPNotify(s *server.Server, log *log.Logger) *IMAPNotify {
return &IMAPNotify{
server: s,
log: log,
}
}()
}

37
internal/imapserver/user.go
View File

@ -9,35 +9,38 @@
package imapserver
import (
"encoding/hex"
"errors"
"fmt"
"log"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/backend"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/storage"
)
type User struct {
backend *Backend
//backend *Backend
storage storage.Storage
username string
session *account.Session
conn *imap.ConnInfo
log *log.Logger
}
func (u *User) Username() string {
return hex.EncodeToString(u.backend.Config.PublicKey)
return u.session.Username()
}
func (u *User) ListMailboxes(subscribed bool) (mailboxes []backend.Mailbox, err error) {
names, err := u.backend.Storage.MailboxList(subscribed)
names, err := u.storage.MailboxList(u.username, subscribed)
if err != nil {
return nil, err
}
for _, mailbox := range names {
mailboxes = append(mailboxes, &Mailbox{
backend: u.backend,
storage: u.storage,
user: u,
name: mailbox,
})
@ -49,17 +52,17 @@ func (u *User) ListMailboxes(subscribed bool) (mailboxes []backend.Mailbox, err
func (u *User) GetMailbox(name string) (mailbox backend.Mailbox, err error) {
if name == "" {
return &Mailbox{
backend: u.backend,
storage: u.storage,
user: u,
name: "",
}, nil
}
ok, _ := u.backend.Storage.MailboxSelect(name)
ok, _ := u.storage.MailboxSelect(u.username, name)
if !ok {
return nil, fmt.Errorf("mailbox %q not found", name)
}
return &Mailbox{
backend: u.backend,
storage: u.storage,
user: u,
name: name,
}, nil
@ -68,13 +71,13 @@ func (u *User) GetMailbox(name string) (mailbox backend.Mailbox, err error) {
func (u *User) CreateMailbox(name string) error {
u.log.Printf("Creating mailbox '%s'...\n", name)
if e := u.backend.Storage.MailboxCreate(name); e != nil {
u.log.Printf("Error creating mailbox '%s': %v\n", name, e);
return e;
if e := u.storage.MailboxCreate(u.username, name); e != nil {
u.log.Printf("Error creating mailbox '%s': %v\n", name, e)
return e
}
u.log.Printf("Created mailbox '%s'\n", name);
return nil;
u.log.Printf("Created mailbox '%s'\n", name)
return nil
}
func (u *User) DeleteMailbox(name string) error {
@ -82,12 +85,12 @@ func (u *User) DeleteMailbox(name string) error {
case "INBOX", "Outbox", "Sent":
return errors.New("Cannot delete " + name)
default:
if e := u.backend.Storage.MailboxDelete(name); e != nil {
if e := u.storage.MailboxDelete(u.username, name); e != nil {
u.log.Printf("Error deleting mailbox '%s': %v\n", name, e)
return e;
return e
} else {
u.log.Printf("Deleted mailbox '%s'\n", name)
return e;
return e
}
}
}
@ -97,7 +100,7 @@ func (u *User) RenameMailbox(existingName, newName string) error {
case "INBOX", "Outbox", "Sent":
return errors.New("Cannot rename " + existingName)
default:
return u.backend.Storage.MailboxRename(existingName, newName)
return u.storage.MailboxRename(u.username, existingName, newName)
}
}

137
internal/notify/notify.go Normal file
View File

@ -0,0 +1,137 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package notify
import (
"log"
"sync"
)
type EventType int
const (
EventMailNew EventType = iota
EventMailUpdate
EventMailPurge
EventFlagsUpdate
EventHeartbit
// ...
)
const AllUsers = "*"
type Event interface {
GetType() EventType
GetAccountPK() string
}
type HeartbitEvent struct {
Type EventType
AccountPK string
}
func (e *HeartbitEvent) GetAccountPK() string { return e.AccountPK }
func (e *HeartbitEvent) GetType() EventType { return EventHeartbit }
type EmailEvent struct {
Type EventType
MessageId, Count int64
AccountPK, MailBox string
done chan struct{}
}
func NewEmailEvent(pk, mailbox string, id, count int64) *EmailEvent {
return &EmailEvent{
AccountPK: pk,
MailBox: mailbox,
MessageId: id,
Count: count,
done: make(chan struct{}),
}
}
func (e *EmailEvent) GetAccountPK() string { return e.AccountPK }
func (e *EmailEvent) GetType() EventType { return e.Type }
// imap/backend/update interface
func (e *EmailEvent) Username() string { return e.AccountPK }
func (e *EmailEvent) Mailbox() string { return e.MailBox }
func (e *EmailEvent) Done() chan struct{} { return e.done }
type FlagsEvent struct {
AccountPK string
Mailbox string
MessageID int64
Seen bool
Answered bool
Flagged bool
Deleted bool
}
func (e *FlagsEvent) GetAccountPK() string { return e.AccountPK }
func (e *FlagsEvent) GetType() EventType { return EventFlagsUpdate }
type Notify struct {
log *log.Logger
mu sync.Mutex
listeners map[string]map[chan Event]struct{}
}
func NewNotifyBus(log *log.Logger) *Notify {
return &Notify{
log: log,
}
}
// Subscribe for any methods
func (n *Notify) Subscribe(pk string) chan Event {
n.mu.Lock()
defer n.mu.Unlock()
if n.listeners == nil {
n.listeners = make(map[string]map[chan Event]struct{})
}
if n.listeners[pk] == nil {
n.listeners[pk] = make(map[chan Event]struct{})
}
ch := make(chan Event, 10)
n.listeners[pk][ch] = struct{}{}
return ch
}
func (n *Notify) Unsubscribe(pk string, ch chan Event) {
n.mu.Lock()
defer n.mu.Unlock()
if subs, ok := n.listeners[pk]; ok {
delete(subs, ch)
if len(subs) == 0 {
delete(n.listeners, pk)
}
}
}
// Do Notification for clients
func (n *Notify) NotifyNew(pk string, event Event) error {
n.mu.Lock()
defer n.mu.Unlock()
go func() {
if subs, ok := n.listeners[pk]; ok {
for ch := range subs {
ch <- event
}
}
if subs, ok := n.listeners[AllUsers]; ok {
for ch := range subs {
ch <- event
}
}
}()
return nil
}

78
internal/notify/remote/remote.go Normal file
View File

@ -0,0 +1,78 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"crypto/ed25519"
"fmt"
"github.com/neilalexander/yggmail/internal/notify"
"github.com/neilalexander/yggmail/internal/remote"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/transport"
"github.com/neilalexander/yggmail/internal/utils"
)
type RemoteNotifer struct {
Client *remote.Client
targetNode ed25519.PublicKey
userPk ed25519.PublicKey
}
func NewRemoteNotifer(targetNode, userPk ed25519.PublicKey, signer remote.Signer, tr transport.Transport) *RemoteNotifer {
return &RemoteNotifer{
Client: remote.NewClient(userPk, signer, tr),
targetNode: targetNode,
userPk: userPk,
}
}
func (n *RemoteNotifer) Watch(pk, mailbox string) (uint32, uint32, error) {
resp, _, err := n.Client.Call(n.targetNode, &pb.Request{
Call: &pb.Request_MailWatch{
MailWatch: &pb.WatchRequest{Mailbox: mailbox},
},
}, nil, 0)
if err != nil {
return 0, 0, err
}
if !resp.Success {
return 0, 0, fmt.Errorf("remote error: %s", resp.Error)
}
if resp.Error == "HEARTBIT" {
return 0, 0, nil
}
status := resp.GetMboxStatus()
if status != nil {
return status.UidNext, status.MessagesCount, nil
}
return 0, 0, nil
}
func (n *RemoteNotifer) StartRemoteNotifer(nf *notify.Notify, pk ed25519.PublicKey, mailbox string) {
go func() {
for {
pk := utils.EncodeString(n.userPk)
uidNext, count, err := n.Watch(pk, mailbox)
if err != nil {
continue
}
if uidNext == 0 && count == 0 && err == nil {
// fmt.Println("RPC Notify: HEARTBIT: Pk", pk)
// Heartbit: Update connection
continue
}
if nf != nil {
fmt.Println("RPC Notify: Notify: Pk", pk)
ev := notify.NewEmailEvent(pk, mailbox, int64(uidNext), int64(count))
go nf.NotifyNew(pk, ev)
}
}
}()
}

114
internal/remote/client.go Normal file
View File

@ -0,0 +1,114 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"bufio"
"crypto/ed25519"
"encoding/hex"
"fmt"
"io"
"net"
"time"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/transport"
"github.com/neilalexander/yggmail/internal/utils"
)
type rpcSession struct {
conn net.Conn
buf *bufio.Reader
}
type Client struct {
publicKey ed25519.PublicKey
signer Signer
transport transport.Transport
pool chan *rpcSession
}
func NewClient(pubkey ed25519.PublicKey, signer Signer, tr transport.Transport) *Client {
c := &Client{
publicKey: pubkey,
signer: signer,
transport: tr,
pool: make(chan *rpcSession, 3),
}
return c
}
func (c *Client) getConn(target ed25519.PublicKey) (*rpcSession, error) {
select {
case session := <-c.pool:
return session, nil
default:
conn, err := c.transport.Dial(hex.EncodeToString(target))
if err != nil {
return nil, err
}
session := &rpcSession{
conn: conn,
buf: bufio.NewReader(conn),
}
return session, nil
}
}
func (c *Client) returnConn(sess *rpcSession) {
select {
case c.pool <- sess:
default:
sess.conn.Close()
}
}
func (c *Client) Call(targetNode ed25519.PublicKey, req *pb.Request, payload io.ReadCloser, size uint32) (*pb.Response, *Payload, error) {
for range 2 {
sess, err := c.getConn(targetNode)
if err != nil {
return nil, nil, fmt.Errorf("dial failed: %w", err)
}
req.AccountPk = utils.EncodeString(c.publicKey)
req.Timestamp = time.Now().UnixNano()
if err := SignRequest(req, c.signer); err != nil {
c.returnConn(sess)
return nil, nil, err
}
resp, respPayload, err := c.doRPC(sess, req, payload, size)
if err != nil {
sess.conn.Close()
return nil, nil, fmt.Errorf("rpc failed: %w", err)
}
if respPayload != nil && respPayload.R != nil {
respPayload.R.Close()
}
c.returnConn(sess)
return resp, respPayload, nil
}
return nil, nil, fmt.Errorf("all retries failed")
}
func (c *Client) doRPC(sess *rpcSession, req *pb.Request, payload io.ReadCloser, size uint32) (*pb.Response, *Payload, error) {
p := &Payload{
R: payload,
Size: size,
}
if err := SendFrame(sess.conn, req, p); err != nil {
return nil, nil, err
}
return ReadResponse(sess.buf)
}

292
internal/remote/handlers.go Normal file
View File

@ -0,0 +1,292 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"bytes"
"context"
"encoding/hex"
"io"
"time"
"github.com/neilalexander/yggmail/internal/notify"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/utils"
)
// --- MAILBOXES ---
func (srv *Server) handleMailboxList(ctx context.Context, pk string, call *pb.MailboxListRequest) (*pb.Response, error) {
list, err := srv.storage.MailboxList(pk, call.OnlySubscribed)
if err != nil {
return nil, err
}
return &pb.Response{Success: true, Data: &pb.Response_MboxList{MboxList: &pb.MailboxList{Names: list}}}, nil
}
func (srv *Server) handleMailboxCreate(ctx context.Context, pk string, call *pb.MailboxCreateRequest) (*pb.Response, error) {
err := srv.storage.MailboxCreate(pk, call.Name)
srv.log.Printf("RPC Server:: MailboxCreate: Name: %s: Pk: %s", call.Name, pk)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailboxDelete(ctx context.Context, pk string, call *pb.MailboxDeleteRequest) (*pb.Response, error) {
err := srv.storage.MailboxDelete(pk, call.Name)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailboxRename(ctx context.Context, pk string, call *pb.MailboxRenameRequest) (*pb.Response, error) {
err := srv.storage.MailboxRename(pk, call.OldName, call.NewName)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailboxSubscribe(ctx context.Context, pk string, call *pb.MailboxSubscribeRequest) (*pb.Response, error) {
err := srv.storage.MailboxSubscribe(pk, call.Name, call.Subscribed)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailboxStatus(ctx context.Context, pk string, call *pb.MailboxStatusRequest) (*pb.Response, error) {
exists, err := srv.storage.MailboxSelect(pk, call.Name)
if err != nil || !exists {
return &pb.Response{Success: false, Error: "Mailbox not found"}, nil
}
count, _ := srv.storage.MailCount(pk, call.Name)
unseen, _ := srv.storage.MailUnseen(pk, call.Name)
nextID, _ := srv.storage.MailNextID(pk, call.Name)
return &pb.Response{
Success: true,
Data: &pb.Response_MboxStatus{
MboxStatus: &pb.MailboxStatus{
Name: call.Name, MessagesCount: uint32(count), UnseenCount: uint32(unseen), UidNext: uint32(nextID), UidValidity: 1,
},
},
}, nil
}
// --- MAILS ---
// handleMailFetch processes fetching metadata and optionally streams a single message body.
func (srv *Server) handleMailFetch(ctx context.Context, pk string, call *pb.MailFetchRequest) (*pb.Response, *Payload, error) {
pbList := &pb.MailList{}
var messagePayload *Payload
for _, id := range call.Ids {
seq, mail, err := srv.storage.MailSelect(pk, call.Mailbox, int(id))
if err != nil {
continue
}
pbList.Items = append(pbList.Items, &pb.MailMessage{
Id: uint32(mail.ID),
Seq: uint32(seq),
Date: mail.Date.Unix(),
Seen: mail.Seen,
Answered: mail.Answered,
Flagged: mail.Flagged,
Deleted: mail.Deleted,
Size: uint32(mail.Size),
})
// If fetching a single message, prepare the stream for the body payload
if len(call.Ids) == 1 && mail.Body != nil {
messagePayload = &Payload{
R: mail.Body,
Size: uint32(mail.Size),
}
} else if mail.Body != nil {
mail.Body.Close()
}
}
return &pb.Response{
Success: true,
Data: &pb.Response_MailList{MailList: pbList},
}, messagePayload, nil
}
func (srv *Server) handleMailFlags(ctx context.Context, pk string, call *pb.MailUpdateFlagsRequest) (*pb.Response, error) {
err := srv.storage.MailUpdateFlags(pk, call.Mailbox, int(call.Id), call.Seen, call.Answered, call.Flagged, call.Deleted)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailMove(ctx context.Context, pk string, call *pb.MailMoveRequest) (*pb.Response, error) {
_, err := srv.storage.MailMove(pk, call.Mailbox, int(call.Id), call.DestinationMailbox)
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailDelete(ctx context.Context, pk string, call *pb.MailDeleteRequest) (*pb.Response, error) {
err := srv.storage.MailDelete(pk, call.Mailbox, int(call.Id))
return &pb.Response{Success: err == nil}, err
}
func (srv *Server) handleMailExpunge(ctx context.Context, pk string, call *pb.MailExpungeRequest) (*pb.Response, error) {
err := srv.storage.MailExpunge(pk, call.Mailbox)
return &pb.Response{Success: err == nil}, err
}
// --- QUEUE & EXCHANGE ---
func (srv *Server) handleQueueInsert(ctx context.Context, pk string, call *pb.QueueInsertRequest) (*pb.Response, error) {
err := srv.storage.QueueInsertDestinationForID(pk, call.Destination, int(call.Id), call.From, call.Rcpt)
return &pb.Response{Success: err == nil}, err
}
// handleMailPush handles incoming mail streams and routes them (Incoming, Outgoing, or Save).
func (srv *Server) handleMailPush(ctx context.Context, senderPkStr string, call *pb.MailPushRequest, payload *Payload) (*pb.Response, error) {
if payload == nil || payload.R == nil {
return &pb.Response{Success: false, Error: "Empty payload"}, nil
}
defer payload.R.Close()
senderPkBytes, _ := utils.DecodeString(senderPkStr)
fromDom, fromAcc, _ := utils.ParseAddress(call.From)
toDom, toAcc, _ := utils.ParseAddress(call.To)
isFromMe := bytes.Equal(senderPkBytes, fromAcc) && fromDom.Equal(srv.nodePublickey)
isFromPeerServer := bytes.Equal(senderPkBytes, fromDom)
isToMe := toDom.Equal(srv.nodePublickey)
if isFromMe && call.Mailbox != "" {
return srv.saveToFolder(fromAcc, toDom, call.Mailbox, payload.R, payload.Size)
}
if isFromMe {
return srv.processOutgoing(fromAcc, toDom, payload.R, payload.Size)
}
if isFromPeerServer && isToMe {
accTo, _ := srv.accMgr.GetAccount(utils.EncodeString(toAcc))
if accTo != nil {
return srv.processIncoming(accTo.PublicKey, "INBOX", payload.R, payload.Size)
}
}
return &pb.Response{Success: false, Error: "Relay access denied"}, nil
}
func (srv *Server) processIncoming(accPK, mailbox string, content io.Reader, size uint32) (*pb.Response, error) {
id, err := srv.storage.MailCreate(accPK, mailbox, content, int(size))
if err != nil {
return nil, err
}
if count, err := srv.storage.MailCount(accPK, mailbox); err == nil {
srv.notify.NotifyNew(accPK, notify.NewEmailEvent(accPK, mailbox, int64(id), int64(count)))
}
srv.log.Printf("RPC Server: MailPush: Incoming: Mailbox: %s for: %s\n", mailbox, accPK)
return &pb.Response{Success: true, Data: &pb.Response_IdResult{IdResult: uint32(id)}}, nil
}
func (srv *Server) processOutgoing(accPK []byte, targetDom []byte, content io.Reader, size uint32) (*pb.Response, error) {
pkStr := utils.EncodeString(accPK)
_ = srv.storage.MailboxCreate(pkStr, "Outbox")
id, err := srv.storage.MailCreate(pkStr, "Outbox", content, int(size))
if err != nil {
return nil, err
}
srv.log.Printf("RPC Server: MailPush: Outgoing: From: %s\n", accPK)
srv.queue.QueueForServer(hex.EncodeToString(targetDom))
return &pb.Response{Success: true, Data: &pb.Response_IdResult{IdResult: uint32(id)}}, nil
}
func (srv *Server) saveToFolder(accPK []byte, targetDom []byte, mailbox string, content io.Reader, size uint32) (*pb.Response, error) {
pkStr := utils.EncodeString(accPK)
_ = srv.storage.MailboxCreate(pkStr, mailbox)
id, err := srv.storage.MailCreate(pkStr, mailbox, content, int(size))
if err != nil {
return nil, err
}
// Trigger queue processing if saving directly to Outbox
if mailbox == "Outbox" {
srv.queue.QueueForServer(hex.EncodeToString(targetDom))
}
srv.log.Printf("RPC Server: MailPush: Save: Mailbox: %s for: %s\n", mailbox, pkStr)
return &pb.Response{Success: true, Data: &pb.Response_IdResult{IdResult: uint32(id)}}, nil
}
// --- UTILS ---
func (srv *Server) handleMailIdForSeq(ctx context.Context, pk string, call *pb.MailIdForSeqRequest) (*pb.Response, error) {
id, err := srv.storage.MailIDForSeq(pk, call.Mailbox, int(call.Seq))
if err != nil {
return nil, err
}
return &pb.Response{
Success: true,
Data: &pb.Response_IdResult{IdResult: uint32(id)},
}, nil
}
func (srv *Server) handleMailSearch(ctx context.Context, pk string, call *pb.MailSearchRequest) (*pb.Response, error) {
ids, err := srv.storage.MailSearch(pk, call.Mailbox)
if err != nil {
return nil, err
}
pbList := &pb.MailList{}
for _, id := range ids {
pbList.Items = append(pbList.Items, &pb.MailMessage{Id: id})
}
return &pb.Response{
Success: true,
Data: &pb.Response_MailList{MailList: pbList},
}, nil
}
func (srv *Server) handleMailWatch(ctx context.Context, pk string, call *pb.WatchRequest) (*pb.Response, error) {
notifyChan := srv.notify.Subscribe(pk)
defer srv.notify.Unsubscribe(pk, notifyChan)
select {
case ev := <-notifyChan:
return srv.handleNotify(pk, ev)
case <-time.After(40 * time.Second):
return &pb.Response{Success: true, Error: "HEARTBEAT"}, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
func (srv *Server) handleNotify(pk string, event notify.Event) (*pb.Response, error) {
srv.log.Println("RPC: Notify Subscribe: ", pk)
switch ev := event.(type) {
case *notify.EmailEvent:
srv.log.Println("DEBUG:: RPC:: handleNotify:: Send:: EmailEvent: ", pk)
return &pb.Response{
Success: true,
Data: &pb.Response_MboxStatus{
MboxStatus: &pb.MailboxStatus{
MessagesCount: uint32(ev.Count),
UidNext: uint32(ev.MessageId),
},
},
}, nil
case *notify.FlagsEvent:
return &pb.Response{
Success: true,
Data: &pb.Response_MboxStatus{
MboxStatus: &pb.MailboxStatus{},
},
}, nil
default:
return &pb.Response{Success: true, Error: "HEARTBEAT"}, nil
}
}

275
internal/remote/io.go Normal file
View File

@ -0,0 +1,275 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"bufio"
"bytes"
"encoding/binary"
"fmt"
"io"
"os"
"sync"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"google.golang.org/protobuf/proto"
)
var MagicHeader = []byte{0x59, 0x47, 0x47, 0x21} // "YGG!"
var MemoryLimit uint32 = 1024 * 1024 * 2 // 2 Mb
type Payload struct {
R io.ReadCloser
Size uint32
}
func (p *Payload) Close() error {
if p == nil || p.R == nil {
return nil
}
return p.R.Close()
}
type Tempfile struct {
io.ReadSeeker
file *os.File
size uint32
}
func (f *Tempfile) Len() int64 {
return int64(f.size)
}
func (f *Tempfile) Close() error {
if f.file != nil {
f.file.Close()
return os.Remove(f.file.Name())
}
return nil
}
func ReadRawData(reader io.Reader) (io.ReadCloser, int, error) {
peekBuf := new(bytes.Buffer)
peekBuf.Grow(int(MemoryLimit))
limited := io.LimitReader(reader, int64(MemoryLimit)+1)
written, err := peekBuf.ReadFrom(limited)
if err != nil && err != io.EOF {
return nil, 0, fmt.Errorf("failed to read data: %w", err)
}
if written < int64(MemoryLimit) {
return &Tempfile{
ReadSeeker: bytes.NewReader(peekBuf.Bytes()),
size: uint32(peekBuf.Len())}, peekBuf.Len(), nil
}
// Sew data that we readed and not-read
reader = io.MultiReader(peekBuf, reader)
// Read large mail to temp file
payload, err := os.CreateTemp("", "yggmail-*")
if err != nil {
return nil, 0, fmt.Errorf("failed to create temp-data file: %w", err)
}
size, err := io.Copy(payload, reader)
payload.Seek(0, 0)
return &Tempfile{payload, payload, uint32(size)}, int(size), nil
}
// func ReadRawData(r io.Reader, rawLen uint32) (io.ReadCloser, error) {
// if rawLen == 0 {
// return nil, nil
// }
//
// if rawLen <= MemoryLimit {
// // Write in memory
// buf := bytes.NewBuffer(make([]byte, 0, rawLen))
// _, err := io.CopyN(buf, r, int64(rawLen))
// return io.NopCloser(buf), err
// } else {
// // Write to tmp
// tmp, _ := os.CreateTemp("", "yggmail-payload-*")
// _, err := io.CopyN(tmp, r, int64(rawLen))
// tmp.Seek(0, 0) // Return in ahead
// return &Tempfile{File: tmp}, err
// }
// }
//
// Frame = Magic + uint32(Len Protobuff Meta) + uint32(Len Payload) + Protobuff + Payload
//
// Magic Header
//
// len(Protobuff)
// len()
//
// Protobuff Metadata
// Payload Rawdata
// # SendFrame
//
// Send Magic + uint32(Len Protobuff Meta) + uint32(Len Payload) + Protobuff + Payload
func readWithMagic(br *bufio.Reader, maxJunkBytes int) error {
magicIdx := 0
for range maxJunkBytes {
b, err := br.ReadByte()
if err != nil {
return err
}
if b == MagicHeader[magicIdx] {
magicIdx++
if magicIdx == len(MagicHeader) {
return nil // Found
}
} else {
if b == MagicHeader[0] {
magicIdx = 1
} else {
magicIdx = 0
}
}
}
return fmt.Errorf("sync failed: magic header not found")
}
// # ReadFrame
//
// Magic + uint32(Len Protobuff Meta) + uint32(Len Payload) + Protobuff + Payload
//
// Return Response / Request Reader
//
// RawData Reader
var marshalPool = sync.Pool{
New: func() interface{} {
return make([]byte, 0, 1024*64) // Начинаем с 64КБ
},
}
type limitedReadCloser struct {
io.Reader
io.Closer
}
func ReadFrame(br *bufio.Reader, msg proto.Message) (proto.Message, *Payload, error) {
if err := readWithMagic(br, 1024); err != nil {
return nil, nil, err
}
var lengths [8]byte
if _, err := io.ReadFull(br, lengths[:]); err != nil {
return nil, nil, err
}
protoLen := binary.BigEndian.Uint32(lengths[0:4])
rawLen := binary.BigEndian.Uint32(lengths[4:8])
protoData := make([]byte, protoLen)
if _, err := io.ReadFull(br, protoData); err != nil {
return nil, nil, err
}
if err := proto.Unmarshal(protoData, msg); err != nil {
return nil, nil, err
}
var payload *Payload
if rawLen > 0 {
payload = &Payload{
Size: rawLen,
}
lr := io.LimitReader(br, int64(rawLen))
if rc, ok := lr.(io.ReadCloser); ok {
payload.R = &limitedReadCloser{lr, rc}
} else {
payload.R = io.NopCloser(lr)
}
}
return msg, payload, nil
}
func SendFrame(w io.Writer, msg proto.Message, payload *Payload) error {
if payload != nil && payload.R != nil {
defer payload.R.Close()
}
buf := marshalPool.Get().([]byte)
buf = buf[:0]
opts := proto.MarshalOptions{}
data, err := opts.MarshalAppend(buf, msg)
if err != nil {
marshalPool.Put(buf)
return err
}
defer marshalPool.Put(data)
pSize := uint32(0)
if payload != nil {
pSize = payload.Size
}
var header [12]byte
copy(header[0:4], MagicHeader)
binary.BigEndian.PutUint32(header[4:8], uint32(len(data)))
binary.BigEndian.PutUint32(header[8:12], pSize)
if _, err := w.Write(header[:]); err != nil {
return err
}
if _, err := w.Write(data); err != nil {
return err
}
if payload != nil && payload.Size > 0 {
if _, err := io.CopyN(w, payload.R, int64(pSize)); err != nil {
return err
}
}
return nil
}
func ReadRequest(r *bufio.Reader) (*pb.Request, *Payload, error) {
req := &pb.Request{}
_, payload, err := ReadFrame(r, req)
if err != nil {
return nil, nil, fmt.Errorf("ReadRequest: %w", err)
}
if payload == nil {
return req, nil, nil
}
payload.R, _, err = ReadRawData(payload.R)
if err != nil {
return nil, nil, fmt.Errorf("Error read payload")
}
return req, payload, nil
}
func ReadResponse(r *bufio.Reader) (*pb.Response, *Payload, error) {
resp := &pb.Response{}
_, payload, err := ReadFrame(r, resp)
if err != nil {
return nil, nil, fmt.Errorf("ReadRequest: %w", err)
}
// Unmarshal Protobuff metadata
// Read File
if payload == nil {
return resp, nil, nil
}
payload.R, _, err = ReadRawData(payload.R)
if err != nil {
return nil, nil, fmt.Errorf("Error read payload")
}
return resp, payload, nil
}

210
internal/remote/server.go Normal file
View File

@ -0,0 +1,210 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"bufio"
"context"
"crypto/ed25519"
"fmt"
"io"
"log"
"net"
"time"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/notify"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/utils"
)
type Server struct {
log *log.Logger
notify Notify
queue Queues
accMgr *account.Manager
storage storage.Storage
nodePublickey ed25519.PublicKey
validator *ReqValidator
}
type Notify interface {
Subscribe(pbk string) chan notify.Event
Unsubscribe(pbk string, ch chan notify.Event)
NotifyNew(pk string, ev notify.Event) error
}
type Queues interface {
QueueForServer(serverHex string) error
}
func NewRemoteServer(nodePublickey ed25519.PublicKey, s storage.Storage, m *account.Manager, l *log.Logger, notify Notify, queue Queues) *Server {
return &Server{
storage: s,
accMgr: m,
log: l,
notify: notify,
queue: queue,
nodePublickey: nodePublickey,
validator: NewValidator(time.Minute * 3),
}
}
func (srv *Server) Serve(l net.Listener) error {
srv.log.Println("Unified Remote Storage RPC server started (Streaming Mode)")
for {
conn, err := l.Accept()
if err != nil {
continue
}
// Serving in new gorutine
go srv.handleSession(conn)
}
}
func (srv *Server) handleSession(conn net.Conn) {
defer conn.Close()
br := bufio.NewReader(conn)
for {
conn.SetDeadline(time.Now().Add(30 * time.Second))
req, reqPayload, err := ReadRequest(br)
if err != nil {
if err != io.EOF {
srv.log.Printf("RPC: Session error: %v", err)
}
return
}
func() {
conn.SetDeadline(time.Now().Add(2 * time.Minute))
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(2*time.Minute))
defer cancel()
resp, respPayload := srv.handleRequest(ctx, req, reqPayload)
err = SendFrame(conn, resp, respPayload)
if reqPayload != nil {
io.Copy(io.Discard, reqPayload.R)
reqPayload.Close()
}
if respPayload != nil {
respPayload.Close()
}
if err != nil {
srv.log.Printf("RPC: Send error: %v", err)
return
}
}()
}
}
func RespErr(err error) *pb.Response {
return &pb.Response{Success: false, Error: err.Error()}
}
func (srv *Server) handleRequest(ctx context.Context, req *pb.Request, reqPayload *Payload) (resp *pb.Response, respPayload *Payload) {
// srv.log.Printf("RPC: Request received from %s", req.AccountPk)
sPk, err := utils.DecodeString(req.AccountPk)
if err != nil {
return RespErr(fmt.Errorf("Error decode key")), respPayload
}
// if ok, err := VerifyRequest(req, sPk); err != nil || !ok {
// return nil, fmt.Errorf("401 Unauthorized: Signature mismatch")
// }
// srv.log.Printf("RPC: Request Verifed from %s", req.AccountPk)
ok, err := srv.validator.Validate(req, sPk)
if err != nil {
return RespErr(fmt.Errorf("401 Unauthorized: Signature mismatch")), respPayload
}
if !ok {
return &pb.Response{Success: true, Error: "duplicate request ignored"}, respPayload
}
// Check if this AccountPubKey belongs to a local account we manage
acc, _ := srv.accMgr.GetAccount(req.AccountPk)
isLocalAccount := (acc != nil)
_, isPushCall := req.Call.(*pb.Request_MailPush)
if !isLocalAccount && !isPushCall {
return RespErr(fmt.Errorf("403 Forbidden: Remote management not allowed")), respPayload
}
var rcname string
switch call := req.Call.(type) {
// --- Mailbox management (Local Account only via Policy) ---
case *pb.Request_MboxList:
resp, err = srv.handleMailboxList(ctx, req.AccountPk, call.MboxList)
rcname = "MboxList"
case *pb.Request_MboxCreate:
resp, err = srv.handleMailboxCreate(ctx, req.AccountPk, call.MboxCreate)
rcname = "MboxCreate"
case *pb.Request_MboxDelete:
resp, err = srv.handleMailboxDelete(ctx, req.AccountPk, call.MboxDelete)
rcname = "MboxDelete"
case *pb.Request_MboxRename:
resp, err = srv.handleMailboxRename(ctx, req.AccountPk, call.MboxRename)
rcname = "MboxRename"
case *pb.Request_MboxSub:
resp, err = srv.handleMailboxSubscribe(ctx, req.AccountPk, call.MboxSub)
rcname = "MboxSub"
case *pb.Request_MboxStatus:
resp, err = srv.handleMailboxStatus(ctx, req.AccountPk, call.MboxStatus)
rcname = "MboxStatus"
// --- Mail management ---
case *pb.Request_MailPush:
// handleMailPush has its own logic for Case A (Local) and Case B (Remote Node)
resp, err = srv.handleMailPush(ctx, req.AccountPk, call.MailPush, reqPayload)
rcname = "MailPush"
case *pb.Request_MailFetch:
resp, respPayload, err = srv.handleMailFetch(ctx, req.AccountPk, call.MailFetch)
rcname = "MailFetch"
case *pb.Request_MailFlags:
resp, err = srv.handleMailFlags(ctx, req.AccountPk, call.MailFlags)
rcname = "MailFlags"
case *pb.Request_MailMove:
resp, err = srv.handleMailMove(ctx, req.AccountPk, call.MailMove)
rcname = "MailMove"
case *pb.Request_MailDel:
resp, err = srv.handleMailDelete(ctx, req.AccountPk, call.MailDel)
rcname = "MailDel"
case *pb.Request_MailExpunge:
resp, err = srv.handleMailExpunge(ctx, req.AccountPk, call.MailExpunge)
rcname = "MailExpunge"
case *pb.Request_MailIdForSeq:
resp, err = srv.handleMailIdForSeq(ctx, req.AccountPk, call.MailIdForSeq)
rcname = "MailIdForSeq"
case *pb.Request_MailSearch:
resp, err = srv.handleMailSearch(ctx, req.AccountPk, call.MailSearch)
rcname = "MailSearch"
case *pb.Request_MailWatch:
resp, err = srv.handleMailWatch(ctx, req.AccountPk, call.MailWatch)
// --- Queue management (Local Account only via Policy) ---
case *pb.Request_QueueInsert:
resp, err = srv.handleQueueInsert(ctx, req.AccountPk, call.QueueInsert)
rcname = "QueueInsert"
default:
rcname = "Not Implemented"
resp = &pb.Response{Success: false, Error: "501 Not Implemented"}
}
// srv.log.Printf("RPC: %s : Authorized: %t: Publickey: %s :", rcname, isLocalAccount, req.AccountPk)
if err != nil {
srv.log.Printf("RPC Error: %s %v", rcname, err)
resp = &pb.Response{Success: false, Error: err.Error()}
}
return resp, respPayload
}

111
internal/remote/sign.go Normal file
View File

@ -0,0 +1,111 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"crypto/ed25519"
"fmt"
"sync"
"time"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/utils"
"google.golang.org/protobuf/proto"
)
type ReqValidator struct {
r map[string]time.Time
m sync.RWMutex
window time.Duration
}
func NewValidator(window time.Duration) *ReqValidator {
r := &ReqValidator{
window: window,
r: map[string]time.Time{},
}
go r.Cleaner()
return r
}
func (rv *ReqValidator) Validate(req *pb.Request, pub ed25519.PublicKey) (bool, error) {
diff := time.Since(time.Unix(0, req.Timestamp))
if diff < 0 {
diff = -diff
}
if diff > rv.window {
return false, fmt.Errorf("request timestamp is too old or too far in future")
}
ok, err := VerifyRequest(req, pub)
if err != nil || !ok {
return false, err
}
rv.m.Lock()
defer rv.m.Unlock()
if _, exist := rv.r[utils.EncodeString(req.Signature)]; exist {
return false, nil
}
rv.r[utils.EncodeString(req.Signature)] = time.Now()
return true, nil
}
func (rv *ReqValidator) Cleaner() {
for tn := range time.NewTicker(time.Minute).C {
rv.clean(tn)
}
}
func (rv *ReqValidator) clean(tn time.Time) {
rv.m.Lock()
defer rv.m.Unlock()
for s, tr := range rv.r {
if tr.Add(rv.window).Before(tn) {
delete(rv.r, s)
}
}
}
type Signer func([]byte) ([]byte, error)
func PrepareToSign(req *pb.Request) ([]byte, error) {
clone := proto.Clone(req).(*pb.Request)
clone.Signature = nil
opts := proto.MarshalOptions{Deterministic: true}
return opts.Marshal(clone)
}
func SignRequest(req *pb.Request, signer Signer) error {
data, err := PrepareToSign(req)
if err != nil {
return err
}
req.Signature, err = signer(data)
if err != nil {
return err
}
return nil
}
func VerifyRequest(req *pb.Request, pubKey ed25519.PublicKey) (bool, error) {
if len(req.Signature) == 0 {
return false, fmt.Errorf("missing signature")
}
data, err := PrepareToSign(req)
if err != nil {
return false, err
}
if !ed25519.Verify(pubKey, data, req.Signature) {
return false, fmt.Errorf("cryptographic signature mismatch")
}
return true, nil
}

1534
internal/remote/types/request.pb.go Normal file
View File

File diff suppressed because it is too large Load Diff

130
internal/remote/types/request.proto Normal file
View File

@ -0,0 +1,130 @@
syntax = "proto3";
package types;
option go_package = "github.com/neilalexander/yggmail/internal/remote/types";
message Request {
uint32 version = 1
string account_pk = 2;
bytes signature = 3;
int64 timestamp = 4;
oneof call {
// Core
AccountGetRequest account_get = 5;
AccountDeleteRequest account_delete = 6;
// Mailboxes
MailboxListRequest mbox_list = 7;
MailboxCreateRequest mbox_create = 8;
MailboxDeleteRequest mbox_delete = 9;
MailboxRenameRequest mbox_rename = 10;
MailboxSubscribeRequest mbox_sub = 11;
MailboxStatusRequest mbox_status = 12;
// Messages
MailPushRequest mail_push = 13;
MailFetchRequest mail_fetch = 14;
MailSearchRequest mail_search = 15;
MailUpdateFlagsRequest mail_flags = 16;
MailMoveRequest mail_move = 17;
MailDeleteRequest mail_del = 18;
MailExpungeRequest mail_expunge = 19;
MailIdForSeqRequest mail_id_for_seq = 20;
// Queue
QueueInsertRequest queue_insert = 22;
// Notify
WatchRequest mail_watch = 23;
}
}
message AccountGetRequest {}
message AccountDeleteRequest {}
message MailboxListRequest {
bool only_subscribed = 1;
}
message MailboxCreateRequest {
string name = 1;
}
message MailboxDeleteRequest {
string name = 1;
}
message MailboxRenameRequest {
string old_name = 1;
string new_name = 2;
}
message MailboxSubscribeRequest {
string name = 1;
bool subscribed = 2;
}
message MailboxStatusRequest {
string name = 1;
}
message MailPushRequest {
string from = 1;
string to = 2;
string mailbox = 4;
}
message MailFetchRequest {
string mailbox = 1;
repeated uint32 ids = 2;
uint32 offset = 3;
uint32 limit = 4;
}
message MailSearchRequest {
string mailbox = 1;
string criteria = 2;
}
message MailUpdateFlagsRequest {
string mailbox = 1;
uint32 id = 2;
bool seen = 3;
bool answered = 4;
bool flagged = 5;
bool deleted = 6;
string op = 7;
}
message MailMoveRequest {
string mailbox = 1;
uint32 id = 2;
string destination_mailbox = 3;
}
message MailDeleteRequest {
string mailbox = 1;
uint32 id = 2;
}
message MailExpungeRequest {
string mailbox = 1;
}
message MailIdForSeqRequest {
string mailbox = 1;
uint32 seq = 2;
}
message QueueInsertRequest {
string destination = 1;
uint32 id = 2;
string from = 3;
string rcpt = 4;
}
message WatchRequest {
string mailbox = 1;
}

606
internal/remote/types/response.pb.go Normal file
View File

@ -0,0 +1,606 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.1
// source: internal/remote/types/response.proto
package types
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type Response struct {
state protoimpl.MessageState `protogen:"open.v1"`
Success bool `protobuf:"varint,1,opt,name=success,proto3" json:"success,omitempty"`
Error string `protobuf:"bytes,2,opt,name=error,proto3" json:"error,omitempty"`
// Types that are valid to be assigned to Data:
//
// *Response_Account
// *Response_MboxList
// *Response_MboxStatus
// *Response_MailList
// *Response_IdResult
// *Response_BoolResult
Data isResponse_Data `protobuf_oneof:"data"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Response) Reset() {
*x = Response{}
mi := &file_internal_remote_types_response_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Response) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Response) ProtoMessage() {}
func (x *Response) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Response.ProtoReflect.Descriptor instead.
func (*Response) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{0}
}
func (x *Response) GetSuccess() bool {
if x != nil {
return x.Success
}
return false
}
func (x *Response) GetError() string {
if x != nil {
return x.Error
}
return ""
}
func (x *Response) GetData() isResponse_Data {
if x != nil {
return x.Data
}
return nil
}
func (x *Response) GetAccount() *AccountInfo {
if x != nil {
if x, ok := x.Data.(*Response_Account); ok {
return x.Account
}
}
return nil
}
func (x *Response) GetMboxList() *MailboxList {
if x != nil {
if x, ok := x.Data.(*Response_MboxList); ok {
return x.MboxList
}
}
return nil
}
func (x *Response) GetMboxStatus() *MailboxStatus {
if x != nil {
if x, ok := x.Data.(*Response_MboxStatus); ok {
return x.MboxStatus
}
}
return nil
}
func (x *Response) GetMailList() *MailList {
if x != nil {
if x, ok := x.Data.(*Response_MailList); ok {
return x.MailList
}
}
return nil
}
func (x *Response) GetIdResult() uint32 {
if x != nil {
if x, ok := x.Data.(*Response_IdResult); ok {
return x.IdResult
}
}
return 0
}
func (x *Response) GetBoolResult() bool {
if x != nil {
if x, ok := x.Data.(*Response_BoolResult); ok {
return x.BoolResult
}
}
return false
}
type isResponse_Data interface {
isResponse_Data()
}
type Response_Account struct {
Account *AccountInfo `protobuf:"bytes,3,opt,name=account,proto3,oneof"`
}
type Response_MboxList struct {
MboxList *MailboxList `protobuf:"bytes,4,opt,name=mbox_list,json=mboxList,proto3,oneof"`
}
type Response_MboxStatus struct {
MboxStatus *MailboxStatus `protobuf:"bytes,5,opt,name=mbox_status,json=mboxStatus,proto3,oneof"`
}
type Response_MailList struct {
MailList *MailList `protobuf:"bytes,6,opt,name=mail_list,json=mailList,proto3,oneof"`
}
type Response_IdResult struct {
IdResult uint32 `protobuf:"varint,7,opt,name=id_result,json=idResult,proto3,oneof"`
}
type Response_BoolResult struct {
BoolResult bool `protobuf:"varint,8,opt,name=bool_result,json=boolResult,proto3,oneof"`
}
func (*Response_Account) isResponse_Data() {}
func (*Response_MboxList) isResponse_Data() {}
func (*Response_MboxStatus) isResponse_Data() {}
func (*Response_MailList) isResponse_Data() {}
func (*Response_IdResult) isResponse_Data() {}
func (*Response_BoolResult) isResponse_Data() {}
type AccountInfo struct {
state protoimpl.MessageState `protogen:"open.v1"`
Pubkey string `protobuf:"bytes,1,opt,name=pubkey,proto3" json:"pubkey,omitempty"`
Aliases string `protobuf:"bytes,2,opt,name=aliases,proto3" json:"aliases,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *AccountInfo) Reset() {
*x = AccountInfo{}
mi := &file_internal_remote_types_response_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *AccountInfo) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AccountInfo) ProtoMessage() {}
func (x *AccountInfo) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AccountInfo.ProtoReflect.Descriptor instead.
func (*AccountInfo) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{1}
}
func (x *AccountInfo) GetPubkey() string {
if x != nil {
return x.Pubkey
}
return ""
}
func (x *AccountInfo) GetAliases() string {
if x != nil {
return x.Aliases
}
return ""
}
type MailboxList struct {
state protoimpl.MessageState `protogen:"open.v1"`
Names []string `protobuf:"bytes,1,rep,name=names,proto3" json:"names,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MailboxList) Reset() {
*x = MailboxList{}
mi := &file_internal_remote_types_response_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MailboxList) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MailboxList) ProtoMessage() {}
func (x *MailboxList) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MailboxList.ProtoReflect.Descriptor instead.
func (*MailboxList) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{2}
}
func (x *MailboxList) GetNames() []string {
if x != nil {
return x.Names
}
return nil
}
type MailboxStatus struct {
state protoimpl.MessageState `protogen:"open.v1"`
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
MessagesCount uint32 `protobuf:"varint,2,opt,name=messages_count,json=messagesCount,proto3" json:"messages_count,omitempty"`
UnseenCount uint32 `protobuf:"varint,3,opt,name=unseen_count,json=unseenCount,proto3" json:"unseen_count,omitempty"`
UidNext uint32 `protobuf:"varint,4,opt,name=uid_next,json=uidNext,proto3" json:"uid_next,omitempty"`
UidValidity uint32 `protobuf:"varint,5,opt,name=uid_validity,json=uidValidity,proto3" json:"uid_validity,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MailboxStatus) Reset() {
*x = MailboxStatus{}
mi := &file_internal_remote_types_response_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MailboxStatus) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MailboxStatus) ProtoMessage() {}
func (x *MailboxStatus) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MailboxStatus.ProtoReflect.Descriptor instead.
func (*MailboxStatus) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{3}
}
func (x *MailboxStatus) GetName() string {
if x != nil {
return x.Name
}
return ""
}
func (x *MailboxStatus) GetMessagesCount() uint32 {
if x != nil {
return x.MessagesCount
}
return 0
}
func (x *MailboxStatus) GetUnseenCount() uint32 {
if x != nil {
return x.UnseenCount
}
return 0
}
func (x *MailboxStatus) GetUidNext() uint32 {
if x != nil {
return x.UidNext
}
return 0
}
func (x *MailboxStatus) GetUidValidity() uint32 {
if x != nil {
return x.UidValidity
}
return 0
}
type MailMessage struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id uint32 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Seq uint32 `protobuf:"varint,2,opt,name=seq,proto3" json:"seq,omitempty"`
Date int64 `protobuf:"varint,4,opt,name=date,proto3" json:"date,omitempty"`
Seen bool `protobuf:"varint,5,opt,name=seen,proto3" json:"seen,omitempty"`
Answered bool `protobuf:"varint,6,opt,name=answered,proto3" json:"answered,omitempty"`
Flagged bool `protobuf:"varint,7,opt,name=flagged,proto3" json:"flagged,omitempty"`
Deleted bool `protobuf:"varint,8,opt,name=deleted,proto3" json:"deleted,omitempty"`
Size uint32 `protobuf:"varint,9,opt,name=size,proto3" json:"size,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MailMessage) Reset() {
*x = MailMessage{}
mi := &file_internal_remote_types_response_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MailMessage) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MailMessage) ProtoMessage() {}
func (x *MailMessage) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[4]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MailMessage.ProtoReflect.Descriptor instead.
func (*MailMessage) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{4}
}
func (x *MailMessage) GetId() uint32 {
if x != nil {
return x.Id
}
return 0
}
func (x *MailMessage) GetSeq() uint32 {
if x != nil {
return x.Seq
}
return 0
}
func (x *MailMessage) GetDate() int64 {
if x != nil {
return x.Date
}
return 0
}
func (x *MailMessage) GetSeen() bool {
if x != nil {
return x.Seen
}
return false
}
func (x *MailMessage) GetAnswered() bool {
if x != nil {
return x.Answered
}
return false
}
func (x *MailMessage) GetFlagged() bool {
if x != nil {
return x.Flagged
}
return false
}
func (x *MailMessage) GetDeleted() bool {
if x != nil {
return x.Deleted
}
return false
}
func (x *MailMessage) GetSize() uint32 {
if x != nil {
return x.Size
}
return 0
}
type MailList struct {
state protoimpl.MessageState `protogen:"open.v1"`
Items []*MailMessage `protobuf:"bytes,1,rep,name=items,proto3" json:"items,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MailList) Reset() {
*x = MailList{}
mi := &file_internal_remote_types_response_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MailList) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MailList) ProtoMessage() {}
func (x *MailList) ProtoReflect() protoreflect.Message {
mi := &file_internal_remote_types_response_proto_msgTypes[5]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MailList.ProtoReflect.Descriptor instead.
func (*MailList) Descriptor() ([]byte, []int) {
return file_internal_remote_types_response_proto_rawDescGZIP(), []int{5}
}
func (x *MailList) GetItems() []*MailMessage {
if x != nil {
return x.Items
}
return nil
}
var File_internal_remote_types_response_proto protoreflect.FileDescriptor
const file_internal_remote_types_response_proto_rawDesc = "" +
"\n" +
"$internal/remote/types/response.proto\x12\x05types\"\xd0\x02\n" +
"\bResponse\x12\x18\n" +
"\asuccess\x18\x01 \x01(\bR\asuccess\x12\x14\n" +
"\x05error\x18\x02 \x01(\tR\x05error\x12.\n" +
"\aaccount\x18\x03 \x01(\v2\x12.types.AccountInfoH\x00R\aaccount\x121\n" +
"\tmbox_list\x18\x04 \x01(\v2\x12.types.MailboxListH\x00R\bmboxList\x127\n" +
"\vmbox_status\x18\x05 \x01(\v2\x14.types.MailboxStatusH\x00R\n" +
"mboxStatus\x12.\n" +
"\tmail_list\x18\x06 \x01(\v2\x0f.types.MailListH\x00R\bmailList\x12\x1d\n" +
"\tid_result\x18\a \x01(\rH\x00R\bidResult\x12!\n" +
"\vbool_result\x18\b \x01(\bH\x00R\n" +
"boolResultB\x06\n" +
"\x04data\"?\n" +
"\vAccountInfo\x12\x16\n" +
"\x06pubkey\x18\x01 \x01(\tR\x06pubkey\x12\x18\n" +
"\aaliases\x18\x02 \x01(\tR\aaliases\"#\n" +
"\vMailboxList\x12\x14\n" +
"\x05names\x18\x01 \x03(\tR\x05names\"\xab\x01\n" +
"\rMailboxStatus\x12\x12\n" +
"\x04name\x18\x01 \x01(\tR\x04name\x12%\n" +
"\x0emessages_count\x18\x02 \x01(\rR\rmessagesCount\x12!\n" +
"\funseen_count\x18\x03 \x01(\rR\vunseenCount\x12\x19\n" +
"\buid_next\x18\x04 \x01(\rR\auidNext\x12!\n" +
"\fuid_validity\x18\x05 \x01(\rR\vuidValidity\"\xbb\x01\n" +
"\vMailMessage\x12\x0e\n" +
"\x02id\x18\x01 \x01(\rR\x02id\x12\x10\n" +
"\x03seq\x18\x02 \x01(\rR\x03seq\x12\x12\n" +
"\x04date\x18\x04 \x01(\x03R\x04date\x12\x12\n" +
"\x04seen\x18\x05 \x01(\bR\x04seen\x12\x1a\n" +
"\banswered\x18\x06 \x01(\bR\banswered\x12\x18\n" +
"\aflagged\x18\a \x01(\bR\aflagged\x12\x18\n" +
"\adeleted\x18\b \x01(\bR\adeleted\x12\x12\n" +
"\x04size\x18\t \x01(\rR\x04size\"4\n" +
"\bMailList\x12(\n" +
"\x05items\x18\x01 \x03(\v2\x12.types.MailMessageR\x05itemsB8Z6github.com/neilalexander/yggmail/internal/remote/typesb\x06proto3"
var (
file_internal_remote_types_response_proto_rawDescOnce sync.Once
file_internal_remote_types_response_proto_rawDescData []byte
)
func file_internal_remote_types_response_proto_rawDescGZIP() []byte {
file_internal_remote_types_response_proto_rawDescOnce.Do(func() {
file_internal_remote_types_response_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_internal_remote_types_response_proto_rawDesc), len(file_internal_remote_types_response_proto_rawDesc)))
})
return file_internal_remote_types_response_proto_rawDescData
}
var file_internal_remote_types_response_proto_msgTypes = make([]protoimpl.MessageInfo, 6)
var file_internal_remote_types_response_proto_goTypes = []any{
(*Response)(nil), // 0: types.Response
(*AccountInfo)(nil), // 1: types.AccountInfo
(*MailboxList)(nil), // 2: types.MailboxList
(*MailboxStatus)(nil), // 3: types.MailboxStatus
(*MailMessage)(nil), // 4: types.MailMessage
(*MailList)(nil), // 5: types.MailList
}
var file_internal_remote_types_response_proto_depIdxs = []int32{
1, // 0: types.Response.account:type_name -> types.AccountInfo
2, // 1: types.Response.mbox_list:type_name -> types.MailboxList
3, // 2: types.Response.mbox_status:type_name -> types.MailboxStatus
5, // 3: types.Response.mail_list:type_name -> types.MailList
4, // 4: types.MailList.items:type_name -> types.MailMessage
5, // [5:5] is the sub-list for method output_type
5, // [5:5] is the sub-list for method input_type
5, // [5:5] is the sub-list for extension type_name
5, // [5:5] is the sub-list for extension extendee
0, // [0:5] is the sub-list for field type_name
}
func init() { file_internal_remote_types_response_proto_init() }
func file_internal_remote_types_response_proto_init() {
if File_internal_remote_types_response_proto != nil {
return
}
file_internal_remote_types_response_proto_msgTypes[0].OneofWrappers = []any{
(*Response_Account)(nil),
(*Response_MboxList)(nil),
(*Response_MboxStatus)(nil),
(*Response_MailList)(nil),
(*Response_IdResult)(nil),
(*Response_BoolResult)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_internal_remote_types_response_proto_rawDesc), len(file_internal_remote_types_response_proto_rawDesc)),
NumEnums: 0,
NumMessages: 6,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_internal_remote_types_response_proto_goTypes,
DependencyIndexes: file_internal_remote_types_response_proto_depIdxs,
MessageInfos: file_internal_remote_types_response_proto_msgTypes,
}.Build()
File_internal_remote_types_response_proto = out.File
file_internal_remote_types_response_proto_goTypes = nil
file_internal_remote_types_response_proto_depIdxs = nil
}

51
internal/remote/types/response.proto Normal file
View File

@ -0,0 +1,51 @@
syntax = "proto3";
package types;
option go_package = "github.com/neilalexander/yggmail/internal/remote/types";
message Response {
bool success = 1;
string error = 2;
oneof data {
AccountInfo account = 3;
MailboxList mbox_list = 4;
MailboxStatus mbox_status = 5;
MailList mail_list = 6;
uint32 id_result = 7;
bool bool_result = 8;
}
}
message AccountInfo {
string pubkey = 1;
string aliases = 2;
}
message MailboxList {
repeated string names = 1;
}
message MailboxStatus {
string name = 1;
uint32 messages_count = 2;
uint32 unseen_count = 3;
uint32 uid_next = 4;
uint32 uid_validity = 5;
}
message MailMessage {
uint32 id = 1;
uint32 seq = 2;
int64 date = 4;
bool seen = 5;
bool answered = 6;
bool flagged = 7;
bool deleted = 8;
uint32 size = 9;
}
message MailList {
repeated MailMessage items = 1;
}

203
internal/shell/shell.go Normal file
View File

@ -0,0 +1,203 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package shell
import (
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"log"
"os"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/config"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/utils"
)
type Shell struct {
storage storage.Storage
account *account.Manager
cfg *config.Config
}
func NewShell(storage storage.Storage, log *log.Logger, cfg *config.Config) *Shell {
return &Shell{
storage: storage,
account: account.NewManager(storage, log),
cfg: cfg,
}
}
func passwordProvider() (string, error) {
p, err := account.DefaultProvider("Enter password for your new mail account: ")
if err != nil {
return "", err
}
p2, err := account.DefaultProvider("Comfirm your password: ")
if err != nil {
return "", err
}
if p == p2 {
return p2, nil
}
return "", fmt.Errorf("Password not match")
}
type shellCommand func(args []string) error
func (s *Shell) Run(c string, args []string) {
// fmt.Println(c, args)
var err error
commands := map[string]shellCommand{
"accountAdd": s.accountAdd,
"accountCreateNew": s.accountCreateNew,
"accountGet": s.accountGet,
"accountList": s.accountList,
"accountDelete": s.accountDelete,
"nodeInfo": s.getNodeInfo,
}
cmd, ok := commands[c]
if !ok {
log.Fatal("Command not found!")
os.Exit(1)
}
err = cmd(args)
if err != nil {
fmt.Println(err)
}
}
func (s *Shell) accountAdd(args []string) error {
flags := flag.NewFlagSet("addAccount", flag.ExitOnError)
// All flags
pubkey := flags.String("pub", "", "Account public key")
file := flags.String("f", "", "Added from file")
flags.Parse(args)
var (
err error
acc *types.Account = new(types.Account)
)
if *file != "" {
cont, err := os.ReadFile(*file)
if err != nil {
return err
}
if err := json.Unmarshal(cont, acc); err != nil {
return err
}
if err := s.storage.AccountCreate(acc); err != nil {
return err
}
fmt.Printf("Account %s@%s added", acc.PublicKey, utils.EncodeString(s.cfg.PublicKey))
for _, folder := range []string{"INBOX", "Outbox", "Sent", "Trash", "Drafts"} {
s.storage.MailboxCreate(acc.PublicKey, folder)
}
return nil
}
if acc, err = s.account.AddAccount(*pubkey); err != nil {
return err
}
fmt.Printf("Account %s@%s added", acc.PublicKey, utils.EncodeString(s.cfg.PublicKey))
return nil
}
func (s *Shell) getNodeInfo(args []string) error {
flags := flag.NewFlagSet("nodeInfo", flag.ExitOnError)
flags.Parse(args)
pks := utils.EncodeString(s.cfg.PublicKey)
accList, err := s.storage.AccountList()
if err != nil {
return err
}
fmt.Println("== Node Info == ")
fmt.Println("Address base32:", pks)
fmt.Println("Address hex16:", hex.EncodeToString(s.cfg.PublicKey))
fmt.Println("Accounts: ", len(accList))
return nil
}
func (s *Shell) accountList(args []string) error {
accList, err := s.storage.AccountList()
if err != nil {
return err
}
fmt.Println("Accounts: ", len(accList))
for i, a := range accList {
count, err := s.storage.MailCount(a.PublicKey, "INBOX")
if err != nil {
count = 0
}
fmt.Printf("\n| %d | %s | %s | Mails: %d", i, a.PublicKey, a.Aliases, count)
}
return nil
}
func (s *Shell) accountCreateNew(args []string) error {
a, err := s.account.CreateNewAccount(passwordProvider)
if err != nil {
return err
}
fmt.Printf("Created account: %s@%s", a.PublicKey, utils.EncodeString(s.cfg.PublicKey))
return nil
}
func (s *Shell) accountGet(args []string) error {
flags := flag.NewFlagSet("getAccount", flag.ExitOnError)
// All flags
pubkey := flags.String("pub", "", "Account Publickey")
file := flags.String("o", "", "Write to file")
flags.Parse(args)
var (
err error
acc *types.Account
)
if acc, err = s.account.GetAccount(*pubkey); err != nil {
return err
}
b, err := json.Marshal(acc)
if err != nil {
return err
}
if *file == "" {
fmt.Println(string(b))
} else {
os.WriteFile(*file, b, 0600)
fmt.Println("Write to ", *file)
}
return nil
}
func (s *Shell) accountDelete(args []string) error {
flags := flag.NewFlagSet("getAccount", flag.ExitOnError)
// All flags
pubkey := flags.String("pub", "", "Account Publickey")
flags.Parse(args)
var (
err error
)
if err = s.account.DeleteAccount(*pubkey); err != nil {
return err
}
return nil
}

205
internal/smtpsender/sender.go
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2021 Neil Alexander
* Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -9,18 +9,23 @@
package smtpsender
import (
"crypto/ed25519"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"io"
"log"
"net/mail"
"sync"
"time"
"github.com/emersion/go-smtp"
"github.com/neilalexander/yggmail/internal/config"
"github.com/neilalexander/yggmail/internal/notify"
"github.com/neilalexander/yggmail/internal/remote"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/transport"
"github.com/neilalexander/yggmail/internal/utils"
"go.uber.org/atomic"
@ -31,15 +36,19 @@ type Queues struct {
Log *log.Logger
Transport transport.Transport
Storage storage.Storage
queues sync.Map // servername -> *Queue
RPCClient *remote.Client
Notify *notify.Notify
queues sync.Map
}
func NewQueues(config *config.Config, log *log.Logger, transport transport.Transport, storage storage.Storage) *Queues {
func NewQueues(config *config.Config, log *log.Logger, transport transport.Transport, storage storage.Storage, rpcClient *remote.Client, notify *notify.Notify) *Queues {
qs := &Queues{
Config: config,
Log: log,
Transport: transport,
Storage: storage,
RPCClient: rpcClient,
Notify: notify,
}
time.AfterFunc(time.Second*5, qs.manager)
return qs
@ -51,142 +60,164 @@ func (qs *Queues) manager() {
return
}
for _, destination := range destinations {
_, _ = qs.queueFor(destination)
_ = qs.QueueForServer(destination)
}
time.AfterFunc(time.Minute, qs.manager)
}
func (qs *Queues) QueueFor(from string, rcpts []string, content []byte) error {
pid, err := qs.Storage.MailCreate("Outbox", content)
func (qs *Queues) QueueFor(storage storage.Storage, senderPK string, from string, rcpts []string, m io.Reader, size int) error {
pid, err := storage.MailCreate(senderPK, "Outbox", m, size)
if err != nil {
return fmt.Errorf("q.queues.Storage.MailCreate: %w", err)
return fmt.Errorf("storage.MailCreate(Outbox): %w", err)
}
for _, rcpt := range rcpts {
addr, err := mail.ParseAddress(rcpt)
if err != nil {
return fmt.Errorf("mail.ParseAddress: %w", err)
}
pk, err := utils.ParseAddress(addr.Address)
if err != nil {
return fmt.Errorf("parseAddress: %w", err)
}
host := hex.EncodeToString(pk)
if host == hex.EncodeToString(qs.Config.PublicKey) {
continue
}
if err := qs.Storage.QueueInsertDestinationForID(host, pid, from, rcpt); err != nil {
return fmt.Errorf("qs.Storage.QueueInsertDestinationForID: %w", err)
dPk, _, err := utils.ParseAddress(addr.Address)
if err != nil {
continue
}
_, _ = qs.queueFor(host)
hostHex := hex.EncodeToString(dPk)
err = storage.QueueInsertDestinationForID(senderPK, hostHex, pid, from, rcpt)
if err != nil {
qs.Log.Printf("Failed to insert queue for %s: %v", hostHex, err)
continue
}
if storage == qs.Storage {
_ = qs.QueueForServer(hostHex)
}
}
return nil
}
func (qs *Queues) queueFor(server string) (*Queue, error) {
func (qs *Queues) QueueForServer(server string) error {
v, _ := qs.queues.LoadOrStore(server, &Queue{
queues: qs,
destination: server,
})
q, ok := v.(*Queue)
if !ok {
return nil, fmt.Errorf("type assertion error")
return fmt.Errorf("type assertion error")
}
if q.running.CompareAndSwap(false, true) {
go q.run()
}
return q, nil
return nil
}
type Queue struct {
queues *Queues
// hex16
destination string
running atomic.Bool
}
func (q *Queue) run() {
defer q.running.Store(false)
for {
refs, err := q.queues.Storage.QueueMailIDsForDestination(q.destination)
if err != nil {
q.queues.Log.Println("Error with queue:", err)
if err != nil || len(refs) == 0 {
return
}
defer q.queues.Storage.MailExpunge("Outbox") // nolint:errcheck
for _, ref := range refs {
_, mail, err := q.queues.Storage.MailSelect("Outbox", ref.ID)
_, mail, err := q.queues.Storage.MailSelect(ref.AccountPubKey, "Outbox", ref.ID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
if err = q.queues.Storage.QueueDeleteDestinationForID("Outbox", ref.ID); err != nil {
q.queues.Log.Println("Failed delete queue destination for ID", ref.ID, "due to error:", err)
}
} else {
q.queues.Log.Println("Failed to get mail", ref.ID, "due to error:", err)
}
q.handleError(ref, err)
continue
}
q.queues.Log.Println("Sending mail from", ref.From, "to", q.destination)
q.queues.Log.Printf("Sending mail from %s (Acc: %s) to %s", ref.From, ref.AccountPubKey, q.destination)
if err := func() error {
conn, err := q.queues.Transport.Dial(q.destination)
if err != nil {
return fmt.Errorf("q.queues.Transport.Dial: %w", err)
}
defer conn.Close() // nolint:errcheck
client, err := smtp.NewClient(conn, q.destination)
if err != nil {
return fmt.Errorf("smtp.NewClient: %w", err)
}
defer client.Close() // nolint:errcheck
if err := client.Hello(hex.EncodeToString(q.queues.Config.PublicKey)); err != nil {
q.queues.Log.Println("Remote server", q.destination, "did not accept HELLO:", err)
return fmt.Errorf("client.Hello: %w", err)
deliver := q.deliverRemote
if q.destination == hex.EncodeToString(q.queues.Config.PublicKey) {
deliver = q.deliverLocal
}
if err := client.Mail(ref.From, nil); err != nil {
q.queues.Log.Println("Remote server", q.destination, "did not accept MAIL:", err)
return fmt.Errorf("client.Mail: %w", err)
if err := deliver(ref, mail.Body, mail.Size); err != nil {
q.queues.Log.Printf("Delivery to %s failed: %v. Will retry later.", q.destination, err)
return
}
if err := client.Rcpt(ref.Rcpt); err != nil {
q.queues.Log.Println("Remote server", q.destination, "did not accept RCPT:", err)
return fmt.Errorf("client.Rcpt: %w", err)
}
writer, err := client.Data()
if err != nil {
return fmt.Errorf("client.Data: %w", err)
}
defer writer.Close() // nolint:errcheck
if _, err := writer.Write(mail.Mail); err != nil {
return fmt.Errorf("writer.Write: %w", err)
}
if err := q.queues.Storage.QueueDeleteDestinationForID(q.destination, ref.ID); err != nil {
return fmt.Errorf("q.queues.Storage.QueueDeleteDestinationForID: %w", err)
}
if remaining, err := q.queues.Storage.QueueSelectIsMessagePendingSend("Outbox", ref.ID); err != nil {
return fmt.Errorf("q.queues.Storage.QueueSelectIsMessagePendingSend: %w", err)
} else if !remaining {
q.queues.Log.Printf("Moving mail with id '%d' from Outbox to Sent\n", ref.ID)
return q.queues.Storage.MailMove("Outbox", ref.ID, "Sent")
}
return nil
}(); err != nil {
q.queues.Log.Println("Will retry sending to", q.destination, "later due to error:", err)
// TODO: Send a mail to the inbox on the first instance?
} else {
q.queues.Log.Println("Sent mail from", ref.From, "to", q.destination)
q.finalize(ref)
}
}
}
func (q *Queue) deliverLocal(ref types.QueuedMail, content io.ReadCloser, size uint32) error {
q.queues.Log.Printf("Local delivery for %s", ref.Rcpt)
_, toAcc, err := utils.ParseAddress(ref.Rcpt)
toAccPk := utils.EncodeString(toAcc)
if err != nil {
return fmt.Errorf("local delivery: invalid rcpt address: %w", err)
}
if acc, err := q.queues.Storage.AccountGet(toAccPk); acc == nil && err != nil {
return fmt.Errorf("local delivery: account not found")
}
_ = q.queues.Storage.MailboxCreate(toAccPk, "INBOX")
id, err := q.queues.Storage.MailCreate(toAccPk, "INBOX", content, int(size))
if err != nil {
return fmt.Errorf("local delivery failed: %w", err)
}
if count, err := q.queues.Storage.MailCount(toAccPk, "INBOX"); err == nil {
q.queues.Notify.NotifyNew(toAccPk, notify.NewEmailEvent(toAccPk, "INBOX", int64(id), int64(count)))
}
return nil
}
// Deliver incapsulate SMTP-Transaction
func (q *Queue) deliverRemote(ref types.QueuedMail, content io.ReadCloser, size uint32) error {
dPkb, err := hex.DecodeString(q.destination)
dPk := ed25519.PublicKey(dPkb)
resp, _, err := q.queues.RPCClient.Call(dPk, &pb.Request{
Call: &pb.Request_MailPush{
MailPush: &pb.MailPushRequest{
From: ref.From,
To: ref.Rcpt,
},
},
}, content, size)
if err != nil {
return fmt.Errorf("rpc call failed: %w", err)
}
if !resp.Success {
return fmt.Errorf("remote server returned error: %s", resp.Error)
}
q.queues.Log.Printf("Sender: RPC delivery successful for ID %d to node %s", ref.ID, q.destination)
return nil
}
func (q *Queue) finalize(ref types.QueuedMail) {
// Delete id from Queue
_ = q.queues.Storage.QueueDeleteDestinationForID(ref.AccountPubKey, q.destination, ref.ID)
remaining, err := q.queues.Storage.QueueSelectIsMessagePendingSend(ref.AccountPubKey, "Outbox", ref.ID)
if err == nil && !remaining {
q.queues.Log.Printf("Moving mail %d to Sent for %s", ref.ID, ref.AccountPubKey)
_, _ = q.queues.Storage.MailMove(ref.AccountPubKey, "Outbox", ref.ID, "Sent")
}
_ = q.queues.Storage.MailExpunge(ref.AccountPubKey, "Outbox")
}
func (q *Queue) handleError(ref types.QueuedMail, err error) {
if errors.Is(err, sql.ErrNoRows) {
q.queues.Log.Printf("Cleaning up orphaned queue ref %d for %s", ref.ID, ref.AccountPubKey)
_ = q.queues.Storage.QueueDeleteDestinationForID(ref.AccountPubKey, q.destination, ref.ID)
} else {
q.queues.Log.Printf("Storage error for %s: %v", ref.AccountPubKey, err)
}
}

92
internal/smtpserver/backend.go
View File

@ -9,15 +9,16 @@
package smtpserver
import (
"encoding/hex"
"crypto/ed25519"
"fmt"
"log"
"github.com/emersion/go-smtp"
"github.com/neilalexander/yggmail/internal/config"
"github.com/neilalexander/yggmail/internal/imapserver"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/notify"
"github.com/neilalexander/yggmail/internal/smtpsender"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/remote"
"github.com/neilalexander/yggmail/internal/utils"
)
@ -31,33 +32,47 @@ const (
type Backend struct {
Mode BackendMode
Log *log.Logger
Config *config.Config
Queues *smtpsender.Queues
Storage storage.Storage
Notify *imapserver.IMAPNotify
Notify *notify.Notify
Account *account.Manager
NodePublicKey ed25519.PublicKey
// Config *config.Config
}
func (b *Backend) Login(state *smtp.ConnectionState, username, password string) (smtp.Session, error) {
switch b.Mode {
case BackendModeInternal:
// If our username is email-like, then take just the localpart
if pk, err := utils.ParseAddress(username); err == nil {
if !pk.Equal(b.Config.PublicKey) {
return nil, fmt.Errorf("failed to authenticate: wrong domain in username")
dPk, aPk, err := utils.ParseAddress(username)
if err != nil {
return nil, fmt.Errorf("failed to authenticate: wrong domain or username")
}
// Work in domain mode: Get account
session, err := b.Account.Login(username, password)
if err != nil {
return nil, fmt.Errorf("failed to autheticate: %w", err)
}
username = hex.EncodeToString(b.Config.PublicKey)
// The connection came from our local listener
if authed, err := b.Storage.ConfigTryPassword(password); err != nil {
b.Log.Printf("Failed to authenticate SMTP user %q due to error: %s", username, err)
return nil, fmt.Errorf("failed to authenticate: %w", err)
} else if !authed {
b.Log.Printf("Failed to authenticate SMTP user %q\n", username)
return nil, smtp.ErrAuthRequired
username = utils.EncodeString(aPk)
var storage storage.Storage
if !b.NodePublicKey.Equal(dPk) {
storage = remote.NewRemoteStorage(dPk, session, b.Queues.Transport)
b.Log.Println("SMTP L: Authenticate with remote storage as", username)
//return nil, fmt.Errorf("failed to authenticate: wrong domain in username")
} else {
storage = b.Storage
b.Log.Println("SMTP L: Authenticate with local storage as", username)
}
defer b.Log.Printf("Authenticated SMTP user from %s as %q\n", state.RemoteAddr.String(), username)
defer b.Log.Printf("Authenticated SMTP user as %q\n", username)
return &SessionLocal{
backend: b,
session: session,
storage: storage,
state: state,
}, nil
@ -74,23 +89,34 @@ func (b *Backend) AnonymousLogin(state *smtp.ConnectionState) (smtp.Session, err
return nil, fmt.Errorf("not expecting anonymous connection on internal backend")
case BackendModeExternal:
// The connection came from our overlay listener, so we should check
// that they are who they claim to be
pks, err := hex.DecodeString(state.RemoteAddr.String())
if err != nil {
return nil, fmt.Errorf("hex.DecodeString: %w", err)
}
remote := hex.EncodeToString(pks)
if state.Hostname != remote {
return nil, fmt.Errorf("you are not who you claim to be")
}
return nil, fmt.Errorf("not expecting anonymous connection on smtp. RPC Only")
// // The connection came from our overlay listener, so we should check
// // that they are who they claim to be
b.Log.Println("Incoming SMTP session from", remote)
return &SessionRemote{
backend: b,
state: state,
public: pks[:],
}, nil
// // Transport Layer address. Hex
// nodePk, err := hex.DecodeString(state.RemoteAddr.String())
// if err != nil {
// return nil, fmt.Errorf("hex.DecodeString: %w", err)
// }
// // HELO Hostname. Format
// hostPk, err := utils.DecodeString(state.Hostname)
// if err != nil {
// return nil, fmt.Errorf("utils.DecodeKey: %w", err)
// }
// nPk := ed25519.PublicKey(nodePk)
// hPk := ed25519.PublicKey(hostPk)
// if !nPk.Equal(hPk) {
// return nil, fmt.Errorf("you are not who you claim to be")
// }
// b.Log.Println("SMTP Local: Incoming session from", state.Hostname)
// return &SessionRemote{
// backend: b,
// state: state,
// public: nPk[:],
// }, nil
}
return nil, fmt.Errorf("anonymous login failed")

56
internal/smtpserver/session_local.go
View File

@ -9,20 +9,23 @@
package smtpserver
import (
"bytes"
"encoding/hex"
"crypto/ed25519"
"fmt"
"io"
"time"
"github.com/emersion/go-message"
"github.com/emersion/go-smtp"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/remote"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/utils"
"github.com/neilalexander/yggmail/internal/utils/e2ee"
)
type SessionLocal struct {
backend *Backend
storage storage.Storage
state *smtp.ConnectionState
session *account.Session
from string
rcpt []string
}
@ -30,13 +33,12 @@ type SessionLocal struct {
func (s *SessionLocal) Mail(from string, opts smtp.MailOptions) error {
s.rcpt = s.rcpt[:0]
pk, err := utils.ParseAddress(from)
_, aPk, err := utils.ParseAddress(from)
if err != nil {
return fmt.Errorf("parseAddress: %w", err)
}
if !pk.Equal(s.backend.Config.PublicKey) {
return fmt.Errorf("not allowed to send outgoing mail as %s", from)
if !aPk.Equal(s.session.Publickey()) {
return fmt.Errorf("not allowed to send outgoing mail as account: %s", utils.EncodeString(aPk))
}
s.from = from
@ -49,34 +51,36 @@ func (s *SessionLocal) Rcpt(to string) error {
}
func (s *SessionLocal) Data(r io.Reader) error {
m, err := message.Read(r)
payload, size, err := remote.ReadRawData(io.NopCloser(r))
if err != nil {
return fmt.Errorf("message.Read: %w", err)
return err
}
m.Header.Add(
"Received", fmt.Sprintf("from %s by Yggmail %s; %s",
s.state.RemoteAddr.String(),
hex.EncodeToString(s.backend.Config.PublicKey),
time.Now().String(),
),
)
if !m.Header.Has("Date") {
m.Header.Add(
"Date", time.Now().UTC().Format(time.RFC822),
)
var rcpt []ed25519.PublicKey
for _, r := range s.rcpt {
_, aPk, err := utils.ParseAddress(r)
if err != nil {
s.backend.Log.Println("Data: Parse address: cant parse: ", r)
continue
}
rcpt = append(rcpt, aPk)
}
payload, err = e2ee.EncryptFilter(payload.(io.ReadSeekCloser), s.from, rcpt, s.session.Sign, s.session.Encrypt)
if err != nil {
s.backend.Log.Printf("\nSMTP DATA: error encrypt: %w", err)
return err
}
var b bytes.Buffer
if err := m.WriteTo(&b); err != nil {
return fmt.Errorf("m.WriteTo: %w", err)
payload, size, err = remote.ReadRawData(payload)
if err != nil {
return err
}
if err := s.backend.Queues.QueueFor(s.from, s.rcpt, b.Bytes()); err != nil {
if err := s.backend.Queues.QueueFor(s.storage, s.session.Username(), s.from, s.rcpt, payload, size); err != nil {
return fmt.Errorf("s.backend.Queues.QueueFor: %w", err)
}
s.backend.Log.Println("Queued mail for", s.rcpt)
s.backend.Log.Println("SMTP L: Queued mail for", s.rcpt)
return nil
}

211
internal/smtpserver/session_remote.go
View File

@ -1,98 +1,119 @@
/*
* Copyright (c) 2021 Neil Alexander
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
// /*
// - Copyright (c) 2021 Neil Alexander
// *
// - This Source Code Form is subject to the terms of the Mozilla Public
// - License, v. 2.0. If a copy of the MPL was not distributed with this
// - file, You can obtain one at http://mozilla.org/MPL/2.0/.
// */
package smtpserver
import (
"bytes"
"crypto/ed25519"
"encoding/hex"
"fmt"
"io"
"time"
//
//
// WAS DEPRECATED BUT ARCHIVED FOR RE-IMPLEMENTATION ON PROTOBUFF
//
//
"github.com/emersion/go-message"
"github.com/emersion/go-smtp"
"github.com/neilalexander/yggmail/internal/utils"
)
type SessionRemote struct {
backend *Backend
state *smtp.ConnectionState
public ed25519.PublicKey
from string
}
func (s *SessionRemote) Mail(from string, opts smtp.MailOptions) error {
pk, err := utils.ParseAddress(from)
if err != nil {
return fmt.Errorf("mail.ParseAddress: %w", err)
}
if remote := s.state.RemoteAddr.String(); hex.EncodeToString(pk) != remote {
return fmt.Errorf("not allowed to send incoming mail as %s", from)
}
s.from = from
return nil
}
func (s *SessionRemote) Rcpt(to string) error {
pk, err := utils.ParseAddress(to)
if err != nil {
return fmt.Errorf("mail.ParseAddress: %w", err)
}
if !pk.Equal(s.backend.Config.PublicKey) {
return fmt.Errorf("unexpected recipient for wrong domain")
}
return nil
}
func (s *SessionRemote) Data(r io.Reader) error {
m, err := message.Read(r)
if err != nil {
return fmt.Errorf("message.Read: %w", err)
}
m.Header.Add(
"Received", fmt.Sprintf("from Yggmail %s; %s",
hex.EncodeToString(s.public),
time.Now().String(),
),
)
m.Header.Add(
"Delivery-Date", time.Now().UTC().Format(time.RFC822),
)
var b bytes.Buffer
if err := m.WriteTo(&b); err != nil {
return fmt.Errorf("m.WriteTo: %w", err)
}
if id, err := s.backend.Storage.MailCreate("INBOX", b.Bytes()); err != nil {
return fmt.Errorf("s.backend.Storage.StoreMessageFor: %w", err)
} else {
s.backend.Log.Printf("Stored new mail from %s", s.from)
if count, err := s.backend.Storage.MailCount("INBOX"); err == nil {
if err := s.backend.Notify.NotifyNew(id, count); err != nil {
s.backend.Log.Println("Failed to notify:", s.from)
}
}
}
return nil
}
func (s *SessionRemote) Reset() {}
func (s *SessionRemote) Logout() error {
return nil
}
//
// import (
// "bytes"
// "crypto/ed25519"
// "encoding/hex"
// "fmt"
// "io"
// "time"
//
// "github.com/emersion/go-message"
// "github.com/emersion/go-smtp"
// "github.com/neilalexander/yggmail/internal/notify"
// "github.com/neilalexander/yggmail/internal/utils"
// )
//
// type SessionRemote struct {
// backend *Backend
// state *smtp.ConnectionState
// public ed25519.PublicKey
// from string
// recipient string
// }
//
// func (s *SessionRemote) Mail(from string, opts smtp.MailOptions) error {
// dPk, _, err := utils.ParseAddress(from)
// if err != nil {
// return fmt.Errorf("mail.ParseAddress: %w", err)
// }
// // Tranport layer work on hex16 addresses
// remote, err := hex.DecodeString(s.state.RemoteAddr.String())
//
// rPk := ed25519.PublicKey(remote)
// if err != nil {
// return fmt.Errorf("hex.Decode error decode key")
// }
// if !dPk.Equal(rPk) {
// return fmt.Errorf("not allowed to send incoming mail as %s", from)
// }
//
// s.from = from
// return nil
// }
//
// func (s *SessionRemote) Rcpt(to string) error {
// dPk, aPk, err := utils.ParseAddress(to)
// if err != nil {
// return fmt.Errorf("mail.ParseAddress: %w", err)
// }
//
// if !dPk.Equal(s.backend.NodePublicKey) {
// return fmt.Errorf("unexpected recipient for wrong domain or username")
// }
// if _, err := s.backend.Account.GetAccount(utils.EncodeString(aPk)); err != nil {
// return fmt.Errorf("unexpected recipient for wrong domain or username")
// }
//
// s.recipient = utils.EncodeString(aPk)
// return nil
// }
//
// func (s *SessionRemote) Data(r io.Reader) error {
// m, err := message.Read(r)
// if err != nil {
// return fmt.Errorf("message.Read: %w", err)
// }
//
// // TODO: Verify Message with Header:
// // Header: Sig-ED25519
//
// m.Header.Add(
// "Received", fmt.Sprintf("from Yggmail %s; %s",
// utils.EncodeString(s.public),
// time.Now().String(),
// ),
// )
// m.Header.Add(
// "Delivery-Date", time.Now().UTC().Format(time.RFC822),
// )
//
// var b bytes.Buffer
// if err := m.WriteTo(&b); err != nil {
// return fmt.Errorf("m.WriteTo: %w", err)
// }
//
// if id, err := s.backend.Storage.MailCreate(s.recipient, "INBOX", b.Bytes()); err != nil {
// return fmt.Errorf("s.backend.Storage.StoreMessageFor: %w", err)
// } else {
// s.backend.Log.Printf("Stored new mail from %s", s.from)
//
// if count, err := s.backend.Storage.MailCount(s.recipient, "INBOX"); err == nil {
// if err := s.backend.Notify.NotifyNew(s.recipient, notify.NewEmailEvent(s.recipient, "INBOX", int64(id), int64(count))); err != nil {
// s.backend.Log.Println("SMTP R: Failed to notify:", s.from)
// }
// }
// }
//
// return nil
// }
//
// func (s *SessionRemote) Reset() {}
//
// func (s *SessionRemote) Logout() error {
// return nil
// }

6
internal/smtpserver/smtp.go
View File

@ -10,16 +10,16 @@ package smtpserver
import (
"github.com/emersion/go-smtp"
"github.com/neilalexander/yggmail/internal/imapserver"
"github.com/neilalexander/yggmail/internal/notify"
)
type SMTPServer struct {
server *smtp.Server
backend smtp.Backend
notify *imapserver.IMAPNotify
notify *notify.Notify
}
func NewSMTPServer(backend smtp.Backend, notify *imapserver.IMAPNotify) *SMTPServer {
func NewSMTPServer(backend smtp.Backend, notify *notify.Notify) *SMTPServer {
s := &SMTPServer{
server: smtp.NewServer(backend),
backend: backend,

291
internal/storage/filestore/filestore.go Normal file
View File

@ -0,0 +1,291 @@
/*
* Copyright (c) 2024 JB-SelfCompany / Tyr Contributors
*
* https://github.com/JB-SelfCompany/yggmail
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package filestore
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
// FileStore manages file-based storage for large email messages
type FileStore struct {
basePath string
mu sync.RWMutex
}
// BasePath returns the base path of the file store
func (fs *FileStore) BasePath() string {
fs.mu.RLock()
defer fs.mu.RUnlock()
return fs.basePath
}
// sanitizeMailboxName removes potentially dangerous characters from mailbox names
func sanitizeMailboxName(mailbox string) string {
// Replace path separators and other dangerous characters
sanitized := strings.ReplaceAll(mailbox, "/", "_")
sanitized = strings.ReplaceAll(sanitized, "\\", "_")
sanitized = strings.ReplaceAll(sanitized, "..", "_")
sanitized = strings.ReplaceAll(sanitized, ":", "_")
sanitized = strings.TrimSpace(sanitized)
if sanitized == "" {
sanitized = "default"
}
return sanitized
}
// NewFileStore creates a new FileStore and initializes directory structure
// Creates subdirectories for different mailboxes (INBOX, Outbox, etc.)
func NewFileStore(basePath string) (*FileStore, error) {
if basePath == "" {
return nil, fmt.Errorf("basePath cannot be empty")
}
if err := os.MkdirAll(basePath, 0755); err != nil {
return nil, fmt.Errorf("failed to create base directory: %w", err)
}
return &FileStore{basePath: basePath}, nil
}
func genName(aPk string) (ts int64, hs string) {
ts = time.Now().UnixNano()
h := sha256.New()
h.Write(fmt.Appendf([]byte{}, "%d%s", int(ts), aPk))
hs = hex.EncodeToString(h.Sum(nil))[:16]
return ts, hs
}
// StoreMail stores an email message from a reader to a file
// Returns the file path where the message was stored and the actual bytes written
// Uses atomic write (temp file + rename) to prevent corruption
func (fs *FileStore) StoreMail(aPk string, mailbox string, reader io.Reader) (string, int64, error) {
fs.mu.Lock()
defer fs.mu.Unlock()
sanitized := sanitizeMailboxName(mailbox)
mailboxPath := filepath.Join(fs.basePath, aPk, sanitized)
if err := os.MkdirAll(mailboxPath, 0755); err != nil {
return "", 0, fmt.Errorf("failed to create user mailbox directory: %w", err)
}
ts, hs := genName(aPk)
// Create temp file in same directory (for atomic rename)
tempFile, err := os.CreateTemp(mailboxPath, fmt.Sprintf(".tmp_%d_%s_*", ts, hs))
if err != nil {
return "", 0, fmt.Errorf("failed to create temp file: %w", err)
}
tempPath := tempFile.Name()
//Clean up temp file on error
defer func() {
if tempFile != nil {
tempFile.Close()
os.Remove(tempPath)
}
}()
// Stream data to temp file with buffering
written, err := io.Copy(tempFile, reader)
if err != nil {
return "", 0, fmt.Errorf("failed to write data: %w", err)
}
// Log size for debugging
// fmt.Printf("[FileStore] StoreMail: mailID=%d, written=%d bytes (%.2f MB)\n",
// 0, written, float64(written)/(1024*1024))
// if written == 0 {
// log.Printf("WARNING: empty file written for id=%d", 0)
// }
// Sync to disk
if err := tempFile.Sync(); err != nil {
return "", 0, fmt.Errorf("failed to sync file: %w", err)
}
// Close before rename
if err := tempFile.Close(); err != nil {
return "", 0, fmt.Errorf("failed to close temp file: %w", err)
}
tempFile = nil // Prevent defer cleanup
// Final file path
finalPath := filepath.Join(mailboxPath, fmt.Sprintf("%d_%s.eml", ts, hs))
// Atomic rename
if err := os.Rename(tempPath, finalPath); err != nil {
return "", 0, fmt.Errorf("failed to rename temp file: %w", err)
}
// Return relative path from basePath
relPath, err := filepath.Rel(fs.basePath, finalPath)
if err != nil {
return "", 0, fmt.Errorf("failed to get relative path: %w", err)
}
return relPath, written, nil
}
// ReadMail opens and returns a reader for the specified mail file
// Caller is responsible for closing the returned ReadCloser
func (fs *FileStore) ReadMail(location string) (io.ReadCloser, error) {
fs.mu.RLock()
defer fs.mu.RUnlock()
fullPath := filepath.Join(fs.basePath, location)
if !strings.HasPrefix(filepath.Clean(fullPath), filepath.Clean(fs.basePath)) {
return nil, fmt.Errorf("invalid path traversal attempt")
}
return os.Open(fullPath)
}
// DeleteMail deletes the specified mail file
func (fs *FileStore) DeleteMail(location string) error {
// log.Println("Delete: ", location)
fs.mu.Lock()
defer fs.mu.Unlock()
fullPath := filepath.Join(fs.basePath, location)
return os.Remove(fullPath)
}
// MoveMail moves a mail file to a different mailbox with a new ID
// Returns the new relative path
func (fs *FileStore) MoveMail(oldLocation string, newMailbox string) (string, error) {
fs.mu.Lock()
defer fs.mu.Unlock()
oldFullPath := filepath.Join(fs.basePath, oldLocation)
parts := strings.Split(filepath.ToSlash(oldLocation), "/")
if len(parts) < 3 {
return "", fmt.Errorf("invalid location format")
}
aPk := parts[0]
fileName := parts[len(parts)-1]
sanitizedNew := sanitizeMailboxName(newMailbox)
newRelPath := filepath.Join(aPk, sanitizedNew, fileName)
newFullPath := filepath.Join(fs.basePath, newRelPath)
if err := os.MkdirAll(filepath.Dir(newFullPath), 0755); err != nil {
return "", err
}
if err := os.Rename(oldFullPath, newFullPath); err != nil {
return "", err
}
// log.Printf("Moved file from %s to %s", oldFullPath, newFullPath)
return newRelPath, nil
}
// GetTotalSize returns the total size of all stored mail files in bytes
func (fs *FileStore) GetTotalSize() (int64, error) {
fs.mu.RLock()
defer fs.mu.RUnlock()
var totalSize int64
err := filepath.Walk(fs.basePath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if !info.IsDir() && strings.HasSuffix(info.Name(), ".eml") {
totalSize += info.Size()
}
return nil
})
if err != nil {
return 0, fmt.Errorf("failed to calculate total size: %w", err)
}
return totalSize, nil
}
// CleanupOrphanedFiles removes mail files that are not referenced in the database
// This requires access to the storage layer to query existing mail records
// Returns the number of files deleted and total bytes freed
func (fs *FileStore) CleanupOrphanedFiles(validPaths map[string]bool) (int, int64, error) {
fs.mu.Lock()
defer fs.mu.Unlock()
deletedCount := 0
var deletedSize int64
err := filepath.Walk(fs.basePath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
// Skip directories and non-.eml files
if info.IsDir() || !strings.HasSuffix(info.Name(), ".eml") {
return nil
}
// Get relative path
relPath, err := filepath.Rel(fs.basePath, path)
if err != nil {
return err
}
// Check if this file is in the valid paths map
if !validPaths[relPath] {
// Orphaned file - delete it
size := info.Size()
if err := os.Remove(path); err != nil {
return fmt.Errorf("failed to delete orphaned file %s: %w", relPath, err)
}
deletedCount++
deletedSize += size
}
return nil
})
if err != nil {
return deletedCount, deletedSize, fmt.Errorf("cleanup failed: %w", err)
}
return deletedCount, deletedSize, nil
}
// GetMailboxSize returns the total size of files in a specific mailbox
func (fs *FileStore) GetMailboxSize(aPk string, mailbox string) (int64, error) {
fs.mu.RLock()
defer fs.mu.RUnlock()
sanitized := sanitizeMailboxName(mailbox)
mailboxPath := filepath.Join(fs.basePath, aPk, sanitized)
var totalSize int64
err := filepath.Walk(mailboxPath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if !info.IsDir() && strings.HasSuffix(info.Name(), ".eml") {
totalSize += info.Size()
}
return nil
})
return totalSize, err
}

107
internal/storage/local/storage.go Normal file
View File

@ -0,0 +1,107 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package local
import (
"bytes"
"io"
"log"
"os"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
)
type LocalStorage struct {
storage.Storage
db storage.MailStore
blobs storage.BlobStore
threshold int64
log log.Logger
}
func NewLocalStorage(store storage.MailStore, blobs storage.BlobStore, threshold int64) storage.Storage {
if threshold <= 0 {
threshold = 65536 // Default 64KB
}
return &LocalStorage{
Storage: store,
db: store,
blobs: blobs,
threshold: threshold,
log: *log.New(os.Stdout, "DB :: ", 0555),
}
}
type body struct {
io.ReadCloser
io.Seeker
}
func (l *LocalStorage) MailSelect(pk string, mailbox string, id int) (int, *types.Mail, error) {
seq, mail, err := l.db.MailSelect(pk, mailbox, id)
if err != nil {
return 0, nil, err
}
if mail.MailFile != "" {
l.log.Println("Message Blob large take from ", mail.MailFile)
rc, err := l.blobs.ReadMail(mail.MailFile)
if err != nil {
return 0, nil, err
}
mail.Body = rc
} else {
l.log.Println("Message Blob less take from DB")
br := bytes.NewReader(mail.RawData)
mail.Body = body{io.NopCloser(br), br}
}
return seq, mail, nil
}
func (l *LocalStorage) MailCreate(pk string, mailbox string, r io.Reader, size int) (int, error) {
if int64(size) > l.threshold {
location, written, err := l.blobs.StoreMail(pk, mailbox, r)
if err != nil {
return 0, err
}
l.log.Println("Message Blob large save in ", location)
return l.db.MailCreateRaw(pk, mailbox, nil, location, written)
}
l.log.Println("Message Blob less save in DB")
return l.db.MailCreateRaw(pk, mailbox, r, "", int64(size))
}
func (l *LocalStorage) MailMove(pk string, mailbox string, id int, destination string) (int, error) {
_, mail, err := l.MailSelect(pk, mailbox, id)
if err != nil {
return 0, err
}
id, err = l.db.MailMove(pk, mailbox, id, destination)
if err != nil {
return 0, err
}
if mail.MailFile != "" {
_, err := l.blobs.MoveMail(mail.MailFile, destination)
if err != nil {
return 0, err
}
}
return id, nil
}
func (s *LocalStorage) MailDelete(pk string, mailbox string, id int) error {
log.Println("Delete: ", pk, mailbox, id)
_, mail, err := s.Storage.MailSelect(pk, mailbox, id)
if err == nil && mail.MailFile != "" {
_ = s.blobs.DeleteMail(mail.MailFile)
}
return s.Storage.MailDelete(pk, mailbox, id)
}

211
internal/storage/remote/mail.go Normal file
View File

@ -0,0 +1,211 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"fmt"
"io"
"time"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/utils"
)
// Helpers
func (s *RemoteStorage) getStatus(pk, mailbox string) (*pb.MailboxStatus, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxStatus{
MboxStatus: &pb.MailboxStatusRequest{Name: mailbox},
},
}, nil, 0) // No payload for status
if err != nil {
return nil, err
}
if !resp.Success {
return nil, fmt.Errorf(resp.Error)
}
return resp.GetMboxStatus(), nil
}
// Mail Read
func (s *RemoteStorage) MailCount(pk string, mailbox string) (int, error) {
st, err := s.getStatus(pk, mailbox)
if err != nil {
return 0, err
}
return int(st.MessagesCount), nil
}
func (s *RemoteStorage) MailUnseen(pk string, mailbox string) (int, error) {
st, err := s.getStatus(pk, mailbox)
if err != nil {
return 0, err
}
return int(st.UnseenCount), nil
}
func (s *RemoteStorage) MailNextID(pk string, mailbox string) (int, error) {
st, err := s.getStatus(pk, mailbox)
if err != nil {
return 0, err
}
return int(st.UidNext), nil
}
func (s *RemoteStorage) MailSelect(pk string, mailbox string, id int) (int, *types.Mail, error) {
resp, payload, err := s.Client.Call(s.targetNode, &pb.Request{
Call: &pb.Request_MailFetch{
MailFetch: &pb.MailFetchRequest{
Mailbox: mailbox,
Ids: []uint32{uint32(id)},
},
},
}, nil, 0)
if err != nil {
return 0, nil, fmt.Errorf("rpc select failed: %w", err)
}
if !resp.Success || len(resp.GetMailList().Items) == 0 {
return 0, nil, fmt.Errorf("message not found")
}
m := resp.GetMailList().Items[0]
mail := &types.Mail{
ID: int(m.Id),
Size: m.Size,
Date: time.Unix(m.Date, 0),
Seen: m.Seen,
Answered: m.Answered,
Flagged: m.Flagged,
Deleted: m.Deleted,
}
if payload != nil {
mail.Body = payload.R
}
return int(m.Seq), mail, nil
}
// Mail Create
func (s *RemoteStorage) MailCreate(pk string, mailbox string, content io.Reader, size int) (int, error) {
addr := fmt.Sprintf("%s@%s", pk, utils.EncodeString(s.targetNode))
resp, _, err := s.Client.Call(s.targetNode, &pb.Request{
Call: &pb.Request_MailPush{
MailPush: &pb.MailPushRequest{
From: addr,
To: addr,
Mailbox: mailbox,
},
},
}, io.NopCloser(content), uint32(size))
if err != nil {
return 0, fmt.Errorf("rpc push failed: %w", err)
}
if !resp.Success {
return 0, fmt.Errorf("remote storage error: %s", resp.Error)
}
return int(resp.GetIdResult()), nil
}
func (s *RemoteStorage) MailUpdateFlags(pk string, mailbox string, id int, seen, answered, flagged, deleted bool) error {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailFlags{
MailFlags: &pb.MailUpdateFlagsRequest{
Mailbox: mailbox, Id: uint32(id),
Seen: seen, Answered: answered, Flagged: flagged, Deleted: deleted,
},
},
}, nil, 0)
if err != nil {
return err
}
if !resp.Success {
return fmt.Errorf(resp.Error)
}
return nil
}
// Managements
func (s *RemoteStorage) MailDelete(pk string, mailbox string, id int) error {
_, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailDel{
MailDel: &pb.MailDeleteRequest{Mailbox: mailbox, Id: uint32(id)},
},
}, nil, 0)
return err
}
func (s *RemoteStorage) MailMove(pk string, mailbox string, id int, dest string) (int, error) {
_, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailMove{
MailMove: &pb.MailMoveRequest{
Mailbox: mailbox,
Id: uint32(id),
DestinationMailbox: dest,
},
},
}, nil, 0)
return 0, err
}
func (s *RemoteStorage) MailExpunge(pk string, mailbox string) error {
_, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailExpunge{
MailExpunge: &pb.MailExpungeRequest{Mailbox: mailbox},
},
}, nil, 0)
return err
}
func (s *RemoteStorage) MailIDForSeq(pk string, mailbox string, seq int) (int, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailIdForSeq{
MailIdForSeq: &pb.MailIdForSeqRequest{
Mailbox: mailbox,
Seq: uint32(seq),
},
},
}, nil, 0)
if err != nil {
return 0, err
}
if !resp.Success {
return 0, fmt.Errorf("remote error: %s", resp.Error)
}
return int(resp.GetIdResult()), nil
}
func (s *RemoteStorage) MailSearch(pk string, mailbox string) ([]uint32, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailSearch{
MailSearch: &pb.MailSearchRequest{Mailbox: mailbox},
},
}, nil, 0)
if err != nil {
return nil, err
}
if !resp.Success {
return nil, fmt.Errorf("remote error: %s", resp.Error)
}
var ids []uint32
for _, item := range resp.GetMailList().Items {
ids = append(ids, item.Id)
}
return ids, nil
}

101
internal/storage/remote/mailbox.go Normal file
View File

@ -0,0 +1,101 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"fmt"
pb "github.com/neilalexander/yggmail/internal/remote/types"
)
func (s *RemoteStorage) MailboxList(pk string, onlySubscribed bool) ([]string, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxList{
MboxList: &pb.MailboxListRequest{OnlySubscribed: onlySubscribed},
},
}, nil, 0)
if err != nil {
return nil, err
}
if !resp.Success {
return nil, fmt.Errorf(resp.Error)
}
return resp.GetMboxList().Names, nil
}
func (s *RemoteStorage) MailboxSelect(pk string, mailbox string) (bool, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxStatus{
MboxStatus: &pb.MailboxStatusRequest{Name: mailbox},
},
}, nil, 0)
if err != nil {
return false, err
}
return resp.Success, nil
}
func (s *RemoteStorage) MailboxCreate(pk string, name string) error {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxCreate{
MboxCreate: &pb.MailboxCreateRequest{Name: name},
},
}, nil, 0)
if err != nil {
return err
}
if !resp.Success {
return fmt.Errorf(resp.Error)
}
return nil
}
func (s *RemoteStorage) MailboxDelete(pk string, name string) error {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxDelete{
MboxDelete: &pb.MailboxDeleteRequest{Name: name},
},
}, nil, 0)
if err != nil {
return err
}
if !resp.Success {
return fmt.Errorf(resp.Error)
}
return nil
}
func (s *RemoteStorage) MailboxRename(pk string, old, new string) error {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxRename{
MboxRename: &pb.MailboxRenameRequest{OldName: old, NewName: new},
},
}, nil, 0)
if err != nil {
return err
}
if !resp.Success {
return fmt.Errorf(resp.Error)
}
return nil
}
func (s *RemoteStorage) MailboxSubscribe(pk string, name string, subscribed bool) error {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MboxSub{
MboxSub: &pb.MailboxSubscribeRequest{Name: name, Subscribed: subscribed},
},
}, nil, 0)
if err != nil {
return err
}
if !resp.Success {
return fmt.Errorf(resp.Error)
}
return nil
}

43
internal/storage/remote/queue.go Normal file
View File

@ -0,0 +1,43 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"fmt"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/storage/types"
)
func (s *RemoteStorage) QueueInsertDestinationForID(pk string, dest string, id int, from, rcpt string) error {
_, _, err := s.Call(&pb.Request{
Call: &pb.Request_QueueInsert{
QueueInsert: &pb.QueueInsertRequest{Destination: dest, Id: uint32(id), From: from, Rcpt: rcpt},
},
}, nil, 0)
if err != nil {
return err
}
return nil
}
func (s *RemoteStorage) QueueListDestinations() ([]string, error) { return nil, nil }
func (s *RemoteStorage) QueueMailIDsForDestination(dest string) ([]types.QueuedMail, error) {
return nil, nil
}
func (s *RemoteStorage) QueueDeleteDestinationForID(pk, dest string, id int) error { return nil }
func (s *RemoteStorage) QueueSelectIsMessagePendingSend(pk, mailbox string, id int) (bool, error) {
_, mail, err := s.MailSelect(pk, mailbox, id)
if err != nil {
return false, fmt.Errorf("storage.MailSelect: %w", err)
}
if mail != nil {
return true, nil
}
return false, nil
}

47
internal/storage/remote/storage.go Normal file
View File

@ -0,0 +1,47 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"crypto/ed25519"
"io"
"github.com/neilalexander/yggmail/internal/account"
"github.com/neilalexander/yggmail/internal/remote"
pb "github.com/neilalexander/yggmail/internal/remote/types"
"github.com/neilalexander/yggmail/internal/storage"
"github.com/neilalexander/yggmail/internal/storage/types"
"github.com/neilalexander/yggmail/internal/transport"
)
type RemoteStorage struct {
Client *remote.Client
targetNode ed25519.PublicKey
}
func NewRemoteStorage(targetNode ed25519.PublicKey, sess *account.Session, tr transport.Transport) storage.Storage {
return &RemoteStorage{
Client: remote.NewClient(sess.Publickey(), sess.Sign, tr),
targetNode: targetNode,
}
}
func (s *RemoteStorage) Call(req *pb.Request, r io.ReadCloser, size uint32) (*pb.Response, *remote.Payload, error) {
return s.Client.Call(s.targetNode, req, r, size)
}
func (s *RemoteStorage) ConfigGet(k string) (string, error) { return "", nil }
func (s *RemoteStorage) ConfigSet(k, v string) error { return nil }
func (s *RemoteStorage) ConfigSetPassword(p string) error { return nil }
func (s *RemoteStorage) ConfigTryPassword(p string) (bool, error) { return false, nil }
func (s *RemoteStorage) AccountTryPassword(pk, p string) (bool, error) { return false, nil }
func (s *RemoteStorage) AccountGet(pk string) (*types.Account, error) { return nil, nil }
func (s *RemoteStorage) AccountList() ([]*types.Account, error) { return nil, nil }
func (s *RemoteStorage) AccountCreate(acc *types.Account) error { return nil }
func (s *RemoteStorage) AccountDelete(pk string) error { return nil }

33
internal/storage/remote/watch.go Normal file
View File

@ -0,0 +1,33 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package remote
import (
"fmt"
pb "github.com/neilalexander/yggmail/internal/remote/types"
)
func (s *RemoteStorage) Watch(pk, mailbox string) (uint32, uint32, error) {
resp, _, err := s.Call(&pb.Request{
Call: &pb.Request_MailWatch{
MailWatch: &pb.WatchRequest{Mailbox: mailbox},
},
}, nil, 0)
if err != nil {
return 0, 0, err
}
if !resp.Success {
return 0, 0, fmt.Errorf("remote error: %s", resp.Error)
}
status := resp.GetMboxStatus()
if status != nil {
return status.UidNext, status.MessagesCount, nil
}
return 0, 0, nil
}

9
internal/storage/sqlite3/sqlite3.go
View File

@ -21,12 +21,14 @@ type SQLite3Storage struct {
*TableMailboxes
*TableMails
*TableQueue
*TableAccounts
db *sql.DB
writer *Writer
}
func NewSQLite3StorageStorage(filename string) (*SQLite3Storage, error) {
db, err := sql.Open("sqlite3", "file:"+filename+"?_foreign_keys=on")
db, err := sql.Open("sqlite3", "file:"+filename+"?_foreign_keys=on&mode=rwc&_mmap_size=0")
db.Exec("PRAGMA cache_size = -10000;") // ~10 МБ
if err != nil {
return nil, fmt.Errorf("sql.Open: %w", err)
}
@ -40,6 +42,11 @@ func NewSQLite3StorageStorage(filename string) (*SQLite3Storage, error) {
if err != nil {
return nil, fmt.Errorf("NewTableConfig: %w", err)
}
s.TableAccounts, err = NewTableAccounts(db, s.writer)
if err != nil {
return nil, fmt.Errorf("NewTableAccounts: %w", err)
}
s.TableMailboxes, err = NewTableMailboxes(db, s.writer)
if err != nil {
return nil, fmt.Errorf("NewTableMailboxes: %w", err)

153
internal/storage/sqlite3/table_accounts.go Normal file
View File

@ -0,0 +1,153 @@
/*
* Copyright (c) 2024 Your Name / Yggmail Contributor
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package sqlite3
import (
"database/sql"
"fmt"
"github.com/neilalexander/yggmail/internal/storage/types"
)
type TableAccounts struct {
db *sql.DB
writer *Writer
selectAccount *sql.Stmt
listAccounts *sql.Stmt
createAccount *sql.Stmt
updateAccount *sql.Stmt
deleteAccount *sql.Stmt
}
const accountsSchema = `
CREATE TABLE IF NOT EXISTS accounts (
pubkey TEXT NOT NULL,
aliases TEXT,
passwd_hash TEXT,
private_key TEXT,
PRIMARY KEY(pubkey)
);
`
const accountsSelect = `
SELECT pubkey, aliases, passwd_hash, private_key FROM accounts WHERE pubkey = $1
`
const accountsList = `
SELECT pubkey, aliases, passwd_hash, private_key FROM accounts
`
const accountsCreate = `
INSERT OR IGNORE INTO accounts (pubkey, aliases, passwd_hash, private_key) VALUES($1, $2, $3, $4)
`
const accountsUpdate = `
UPDATE accounts SET aliases = $1, passwd_hash = $2, private_key = $3 WHERE pubkey = $4
`
const accountsDelete = `
DELETE FROM accounts WHERE pubkey = $1
`
func NewTableAccounts(db *sql.DB, writer *Writer) (*TableAccounts, error) {
t := &TableAccounts{
db: db,
writer: writer,
}
_, err := db.Exec(accountsSchema)
if err != nil {
return nil, fmt.Errorf("db.Exec: %w", err)
}
t.selectAccount, err = db.Prepare(accountsSelect)
if err != nil {
return nil, fmt.Errorf("db.Prepare(accountsSelect): %w", err)
}
t.listAccounts, err = db.Prepare(accountsList)
if err != nil {
return nil, fmt.Errorf("db.Prepare(accountsList): %w", err)
}
t.createAccount, err = db.Prepare(accountsCreate)
if err != nil {
return nil, fmt.Errorf("db.Prepare(accountsCreate): %w", err)
}
t.updateAccount, err = db.Prepare(accountsUpdate)
if err != nil {
return nil, fmt.Errorf("db.Prepare(accountsUpdate): %w", err)
}
t.deleteAccount, err = db.Prepare(accountsDelete)
if err != nil {
return nil, fmt.Errorf("db.Prepare(accountsDelete): %w", err)
}
return t, nil
}
func (t *TableAccounts) AccountGet(pubkey string) (*types.Account, error) {
row := t.selectAccount.QueryRow(pubkey)
acc := &types.Account{}
err := row.Scan(&acc.PublicKey, &acc.Aliases, &acc.PasswdHash, &acc.Privatekey)
if err != nil {
if err == sql.ErrNoRows {
return nil, nil // Аккаунт не найден
}
return nil, fmt.Errorf("row.Scan: %w", err)
}
return acc, nil
}
func (t *TableAccounts) AccountList() ([]*types.Account, error) {
rows, err := t.listAccounts.Query()
if err != nil {
return nil, fmt.Errorf("t.listAccounts.Query: %w", err)
}
defer rows.Close()
var accounts []*types.Account
for rows.Next() {
acc := &types.Account{}
if err := rows.Scan(&acc.PublicKey, &acc.Aliases, &acc.PasswdHash, &acc.Privatekey); err != nil {
return nil, fmt.Errorf("rows.Scan: %w", err)
}
accounts = append(accounts, acc)
}
return accounts, nil
}
func (t *TableAccounts) AccountCreate(acc *types.Account) error {
// TODO: Validate that PublicKey is a valid hex before storing
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.createAccount.Exec(acc.PublicKey, acc.Aliases, acc.PasswdHash, acc.Privatekey)
return err
})
}
func (t *TableAccounts) AccountUpdate(acc *types.Account) error {
// TODO: Ensure we don't accidentally overwrite PrivateKey with empty string if not provided in update
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.updateAccount.Exec(acc.Aliases, acc.PasswdHash, acc.Privatekey, acc.PublicKey)
return err
})
}
func (t *TableAccounts) AccountDelete(pubkey string) error {
// TODO: Delete associated mails in TableMails when an account is deleted?
// Or use ON DELETE CASCADE in future schema migrations.
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.deleteAccount.Exec(pubkey)
return err
})
}

64
internal/storage/sqlite3/table_mailboxes.go
View File

@ -1,5 +1,6 @@
/*
* Copyright (c) 2021 Neil Alexander
* Copyright (c) 2024 Your Name / Yggmail Contributor
* (Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -25,40 +26,43 @@ type TableMailboxes struct {
subscribeMailbox *sql.Stmt
}
// Мы изменили схему: PRIMARY KEY теперь составной (аккаунт + папка)
const mailboxesSchema = `
CREATE TABLE IF NOT EXISTS mailboxes (
account_pubkey TEXT NOT NULL,
mailbox TEXT NOT NULL DEFAULT('INBOX'),
subscribed BOOLEAN NOT NULL DEFAULT 1,
PRIMARY KEY(mailbox)
PRIMARY KEY(account_pubkey, mailbox),
FOREIGN KEY(account_pubkey) REFERENCES accounts(pubkey) ON DELETE CASCADE
);
`
const mailboxesList = `
SELECT mailbox FROM mailboxes
SELECT mailbox FROM mailboxes WHERE account_pubkey = $1
`
const mailboxesListSubscribed = `
SELECT mailbox FROM mailboxes WHERE subscribed = 1
SELECT mailbox FROM mailboxes WHERE account_pubkey = $1 AND subscribed = 1
`
const mailboxesSelect = `
SELECT mailbox FROM mailboxes WHERE mailbox = $1
SELECT mailbox FROM mailboxes WHERE account_pubkey = $1 AND mailbox = $2
`
const mailboxesCreate = `
INSERT OR IGNORE INTO mailboxes (mailbox) VALUES($1)
INSERT OR IGNORE INTO mailboxes (account_pubkey, mailbox) VALUES($1, $2)
`
const mailboxesRename = `
UPDATE mailboxes SET mailbox = $1 WHERE mailbox = $2
UPDATE mailboxes SET mailbox = $2 WHERE account_pubkey = $1 AND mailbox = $3
`
const mailboxesDelete = `
DELETE FROM mailboxes WHERE mailbox = $1
DELETE FROM mailboxes WHERE account_pubkey = $1 AND mailbox = $2
`
const mailboxesSubscribe = `
UPDATE mailboxes SET subscribed = $1 WHERE mailbox = $2
UPDATE mailboxes SET subscribed = $2 WHERE account_pubkey = $1 AND mailbox = $3
`
func NewTableMailboxes(db *sql.DB, writer *Writer) (*TableMailboxes, error) {
@ -72,11 +76,11 @@ func NewTableMailboxes(db *sql.DB, writer *Writer) (*TableMailboxes, error) {
}
t.listMailboxes, err = db.Prepare(mailboxesList)
if err != nil {
return nil, fmt.Errorf("db.Prepare(mailboxesCreate): %w", err)
return nil, fmt.Errorf("db.Prepare(mailboxesList): %w", err)
}
t.listMailboxesSubscribed, err = db.Prepare(mailboxesListSubscribed)
if err != nil {
return nil, fmt.Errorf("db.Prepare(mailboxesCreate): %w", err)
return nil, fmt.Errorf("db.Prepare(mailboxesListSubscribed): %w", err)
}
t.selectMailboxes, err = db.Prepare(mailboxesSelect)
if err != nil {
@ -101,16 +105,19 @@ func NewTableMailboxes(db *sql.DB, writer *Writer) (*TableMailboxes, error) {
return t, nil
}
func (t *TableMailboxes) MailboxList(onlySubscribed bool) ([]string, error) {
// Теперь все методы принимают accountPubKey первым аргументом
func (t *TableMailboxes) MailboxList(accountPubKey string, onlySubscribed bool) ([]string, error) {
stmt := t.listMailboxes
if onlySubscribed {
stmt = t.listMailboxesSubscribed
}
rows, err := stmt.Query()
rows, err := stmt.Query(accountPubKey)
if err != nil {
return nil, fmt.Errorf("t.listMailboxes.Query: %w", err)
}
defer rows.Close() // nolint:errcheck
defer rows.Close()
var mailboxes []string
for rows.Next() {
var mailbox string
@ -122,48 +129,47 @@ func (t *TableMailboxes) MailboxList(onlySubscribed bool) ([]string, error) {
return mailboxes, nil
}
func (t *TableMailboxes) MailboxSelect(mailbox string) (bool, error) {
row := t.selectMailboxes.QueryRow(mailbox)
if err := row.Err(); err != nil && err != sql.ErrNoRows {
return false, fmt.Errorf("row.Err: %w", err)
} else if err == sql.ErrNoRows {
func (t *TableMailboxes) MailboxSelect(accountPubKey string, mailbox string) (bool, error) {
row := t.selectMailboxes.QueryRow(accountPubKey, mailbox)
var got string
err := row.Scan(&got)
if err != nil {
if err == sql.ErrNoRows {
return false, nil
}
var got string
if err := row.Scan(&got); err != nil {
return false, fmt.Errorf("row.Scan: %w", err)
}
return mailbox == got, nil
}
func (t *TableMailboxes) MailboxCreate(name string) error {
func (t *TableMailboxes) MailboxCreate(accountPubKey string, name string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.createMailbox.Exec(name)
_, err := t.createMailbox.Exec(accountPubKey, name)
return err
})
}
func (t *TableMailboxes) MailboxRename(old, new string) error {
func (t *TableMailboxes) MailboxRename(accountPubKey string, old, new string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.renameMailbox.Exec(old, new)
_, err := t.renameMailbox.Exec(accountPubKey, new, old)
return err
})
}
func (t *TableMailboxes) MailboxDelete(name string) error {
func (t *TableMailboxes) MailboxDelete(accountPubKey string, name string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.deleteMailbox.Exec(name)
_, err := t.deleteMailbox.Exec(accountPubKey, name)
return err
})
}
func (t *TableMailboxes) MailboxSubscribe(name string, subscribed bool) error {
func (t *TableMailboxes) MailboxSubscribe(accountPubKey string, name string, subscribed bool) error {
sn := 1
if !subscribed {
sn = 0
}
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.subscribeMailbox.Exec(sn, name)
_, err := t.subscribeMailbox.Exec(accountPubKey, sn, name)
return err
})
}

198
internal/storage/sqlite3/table_mails.go
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2021 Neil Alexander
* (Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -9,8 +9,10 @@
package sqlite3
import (
"bytes"
"database/sql"
"fmt"
"io"
"time"
"github.com/neilalexander/yggmail/internal/storage/types"
@ -35,83 +37,88 @@ type TableMails struct {
const mailsSchema = `
CREATE TABLE IF NOT EXISTS mails (
account_pubkey TEXT NOT NULL,
mailbox TEXT NOT NULL,
id INTEGER NOT NULL DEFAULT 1,
mail BLOB NOT NULL,
mail_file TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0,
datetime INTEGER NOT NULL,
seen BOOLEAN NOT NULL DEFAULT 0, -- the mail has been read
answered BOOLEAN NOT NULL DEFAULT 0, -- the mail has been replied to
flagged BOOLEAN NOT NULL DEFAULT 0, -- the mail has been flagged for later attention
deleted BOOLEAN NOT NULL DEFAULT 0, -- the email is marked for deletion at next EXPUNGE
PRIMARY KEY (mailbox, id),
FOREIGN KEY (mailbox) REFERENCES mailboxes(mailbox) ON DELETE CASCADE ON UPDATE CASCADE
seen BOOLEAN NOT NULL DEFAULT 0,
answered BOOLEAN NOT NULL DEFAULT 0,
flagged BOOLEAN NOT NULL DEFAULT 0,
deleted BOOLEAN NOT NULL DEFAULT 0,
PRIMARY KEY (account_pubkey, mailbox, id),
FOREIGN KEY (account_pubkey, mailbox) REFERENCES mailboxes(account_pubkey, mailbox) ON DELETE CASCADE ON UPDATE CASCADE
);
-- View for sequential numbering (important for IMAP FETCH sequence numbers)
CREATE VIEW IF NOT EXISTS inboxes AS SELECT * FROM (
SELECT ROW_NUMBER() OVER (PARTITION BY mailbox) AS seq, * FROM mails
SELECT ROW_NUMBER() OVER (PARTITION BY account_pubkey, mailbox ORDER BY id) AS seq, * FROM mails
)
ORDER BY mailbox, id;
ORDER BY account_pubkey, mailbox, id;
`
const selectMailsStmt = `
SELECT * FROM inboxes
ORDER BY mailbox, id
SELECT seq, id, mail, mail_file, size, datetime, seen, answered, flagged, deleted FROM inboxes
WHERE account_pubkey = $1 AND mailbox = $2
ORDER BY id
`
const selectMailStmt = `
SELECT seq, id, mail, datetime, seen, answered, flagged, deleted FROM inboxes
WHERE mailbox = $1 AND id = $2
ORDER BY mailbox, id
`
const selectMailCountStmt = `
SELECT COUNT(*) FROM mails WHERE mailbox = $1
`
const selectMailUnseenStmt = `
SELECT COUNT(*) FROM mails WHERE mailbox = $1 AND seen = 0
`
const searchMailStmt = `
SELECT id FROM mails
WHERE mailbox = $1
ORDER BY mailbox, id
SELECT seq, id, mail, mail_file, size, datetime, seen, answered, flagged, deleted FROM inboxes
WHERE account_pubkey = $1 AND mailbox = $2 AND id = $3
`
const insertMailStmt = `
INSERT INTO mails (mailbox, id, mail, datetime) VALUES(
$1, (
INSERT INTO mails (account_pubkey, mailbox, id, mail, mail_file, size, datetime) VALUES(
$1, $2, (
SELECT IFNULL(MAX(id)+1,1) AS id FROM mails
WHERE mailbox = $1
), $2, $3
WHERE account_pubkey = $1 AND mailbox = $2
), $3, $4, $5, $6
)
RETURNING id;
`
const selectIDForSeqStmt = `
SELECT id FROM inboxes
WHERE mailbox = $1 AND seq = $2
const selectMailCountStmt = `
SELECT COUNT(*) FROM mails WHERE account_pubkey = $1 AND mailbox = $2
`
const selectMailNextID = `
const selectMailUnseenStmt = `
SELECT COUNT(*) FROM mails WHERE account_pubkey = $1 AND mailbox = $2 AND seen = 0
`
const searchMailStmt = `
SELECT id FROM mails
WHERE account_pubkey = $1 AND mailbox = $2
ORDER BY id
`
const selectIDForSeqStmt = `
SELECT id FROM inboxes
WHERE account_pubkey = $1 AND mailbox = $2 AND seq = $3
`
const selectMailNextIDStmt = `
SELECT IFNULL(MAX(id)+1,1) AS id FROM mails
WHERE mailbox = $1
WHERE account_pubkey = $1 AND mailbox = $2
`
const updateMailFlagsStmt = `
UPDATE mails SET seen = $1, answered = $2, flagged = $3, deleted = $4 WHERE mailbox = $5 AND id = $6
UPDATE mails SET seen = $1, answered = $2, flagged = $3, deleted = $4
WHERE account_pubkey = $5 AND mailbox = $6 AND id = $7
`
const deleteMailStmt = `
UPDATE mails SET deleted = 1 WHERE mailbox = $1 AND id = $2
UPDATE mails SET deleted = 1 WHERE account_pubkey = $1 AND mailbox = $2 AND id = $3
`
const expungeMailStmt = `
DELETE FROM mails WHERE mailbox = $1 AND deleted = 1
DELETE FROM mails WHERE account_pubkey = $1 AND mailbox = $2 AND deleted = 1
`
const moveMailStmt = `
UPDATE mails SET mailbox = $1 WHERE mailbox = $2 AND id = $3
UPDATE mails SET mailbox = $3, id = $4 WHERE account_pubkey = $1 AND mailbox = $2 AND id = $5
`
func NewTableMails(db *sql.DB, writer *Writer) (*TableMails, error) {
@ -123,84 +130,110 @@ func NewTableMails(db *sql.DB, writer *Writer) (*TableMails, error) {
if err != nil {
return nil, fmt.Errorf("db.Exec: %w", err)
}
t.selectMails, err = db.Prepare(selectMailsStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectMailsStmt): %w", err)
return nil, err
}
t.selectMail, err = db.Prepare(selectMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectMailStmt): %w", err)
return nil, err
}
t.selectMailNextID, err = db.Prepare(selectMailNextID)
t.selectMailNextID, err = db.Prepare(selectMailNextIDStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectMailNextID): %w", err)
return nil, err
}
t.selectIDForSeq, err = db.Prepare(selectIDForSeqStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectPIDForIDStmt): %w", err)
return nil, err
}
t.searchMail, err = db.Prepare(searchMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectPIDForIDStmt): %w", err)
return nil, err
}
t.createMail, err = db.Prepare(insertMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(insertMailStmt): %w", err)
return nil, err
}
t.updateMailFlags, err = db.Prepare(updateMailFlagsStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(updateMailSeenStmt): %w", err)
return nil, err
}
t.deleteMail, err = db.Prepare(deleteMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(deleteMailStmt): %w", err)
return nil, err
}
t.expungeMail, err = db.Prepare(expungeMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(expungeMailStmt): %w", err)
return nil, err
}
t.countMails, err = db.Prepare(selectMailCountStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectMailCountStmt): %w", err)
return nil, err
}
t.countUnseenMails, err = db.Prepare(selectMailUnseenStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(selectMailUnseenStmt): %w", err)
return nil, err
}
t.moveMail, err = db.Prepare(moveMailStmt)
if err != nil {
return nil, fmt.Errorf("db.Prepare(moveMailStmt): %w", err)
return nil, err
}
return t, nil
}
func (t *TableMails) MailCreate(mailbox string, data []byte) (int, error) {
// MailRawCreate inserts mail metadata.
// data: content blob (can be nil/empty if stored in mailFile).
// mailFile: relative path in the BlobStore.
// size: pre-calculated size of the full message.
func (t *TableMails) MailCreateRaw(accountPubKey string, mailbox string, data io.Reader, mailFile string, size int64) (int, error) {
var id int
err := t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
return t.createMail.QueryRow(mailbox, data, time.Now().Unix()).Scan(&id)
if data == nil {
data = bytes.NewBuffer(nil)
}
body, err := io.ReadAll(data)
if err != nil {
return fmt.Errorf("MailRawCreate: error read data: %w", err)
}
// Use empty slice instead of nil to satisfy NOT NULL constraint if necessary
if body == nil {
body = []byte{}
}
return t.createMail.QueryRow(accountPubKey, mailbox, body, mailFile, size, time.Now().Unix()).Scan(&id)
})
return id, err
}
func (t *TableMails) MailSelect(mailbox string, id int) (int, *types.Mail, error) {
func (t *TableMails) MailCreate(accountPubKey string, mailbox string, data io.Reader, size int) (int, error) {
panic("sqlite3: MailCreate(stream) must not be called directly. Use LocalStorage wrapper.")
}
// MailSelect retrieves mail metadata.
// Note: The Body field in types.Mail will be populated by the storage coordinator.
func (t *TableMails) MailSelect(accountPubKey string, mailbox string, id int) (int, *types.Mail, error) {
var seq int
var datetime int64
mail := &types.Mail{}
err := t.selectMail.QueryRow(mailbox, id).Scan(
&seq, &mail.ID, &mail.Mail, &datetime,
err := t.selectMail.QueryRow(accountPubKey, mailbox, id).Scan(
&seq, &mail.ID, &mail.RawData, &mail.MailFile, &mail.Size, &datetime,
&mail.Seen, &mail.Answered, &mail.Flagged, &mail.Deleted,
)
if err != nil {
return 0, nil, err
}
mail.Date = time.Unix(datetime, 0)
return seq, mail, err
return seq, mail, nil
}
func (t *TableMails) MailSearch(mailbox string) ([]uint32, error) {
func (t *TableMails) MailSearch(accountPubKey string, mailbox string) ([]uint32, error) {
var ids []uint32
rows, err := t.searchMail.Query(mailbox)
rows, err := t.searchMail.Query(accountPubKey, mailbox)
if err != nil {
return nil, fmt.Errorf("t.searchMail.Query: %w", err)
}
defer rows.Close() // nolint:errcheck
defer rows.Close()
for rows.Next() {
var id uint32
if err := rows.Scan(&id); err != nil {
@ -211,54 +244,59 @@ func (t *TableMails) MailSearch(mailbox string) ([]uint32, error) {
return ids, nil
}
func (t *TableMails) MailNextID(mailbox string) (int, error) {
func (t *TableMails) MailNextID(accountPubKey string, mailbox string) (int, error) {
var id int
err := t.selectMailNextID.QueryRow(mailbox).Scan(&id)
err := t.selectMailNextID.QueryRow(accountPubKey, mailbox).Scan(&id)
return id, err
}
func (t *TableMails) MailIDForSeq(mailbox string, seq int) (int, error) {
func (t *TableMails) MailIDForSeq(accountPubKey string, mailbox string, seq int) (int, error) {
var id int
err := t.selectIDForSeq.QueryRow(mailbox, seq).Scan(&id)
err := t.selectIDForSeq.QueryRow(accountPubKey, mailbox, seq).Scan(&id)
return id, err
}
func (t *TableMails) MailUnseen(mailbox string) (int, error) {
func (t *TableMails) MailUnseen(accountPubKey string, mailbox string) (int, error) {
var unseen int
err := t.countUnseenMails.QueryRow(mailbox).Scan(&unseen)
err := t.countUnseenMails.QueryRow(accountPubKey, mailbox).Scan(&unseen)
return unseen, err
}
func (t *TableMails) MailUpdateFlags(mailbox string, id int, seen, answered, flagged, deleted bool) error {
func (t *TableMails) MailUpdateFlags(accountPubKey string, mailbox string, id int, seen, answered, flagged, deleted bool) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.updateMailFlags.Exec(seen, answered, flagged, deleted, mailbox, id)
_, err := t.updateMailFlags.Exec(seen, answered, flagged, deleted, accountPubKey, mailbox, id)
return err
})
}
func (t *TableMails) MailDelete(mailbox string, id int) error {
func (t *TableMails) MailDelete(accountPubKey string, mailbox string, id int) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.deleteMail.Exec(mailbox, id)
_, err := t.deleteMail.Exec(accountPubKey, mailbox, id)
return err
})
}
func (t *TableMails) MailExpunge(mailbox string) error {
func (t *TableMails) MailExpunge(accountPubKey string, mailbox string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.expungeMail.Exec(mailbox)
_, err := t.expungeMail.Exec(accountPubKey, mailbox)
return err
})
}
func (t *TableMails) MailCount(mailbox string) (int, error) {
func (t *TableMails) MailCount(accountPubKey string, mailbox string) (int, error) {
var count int
err := t.countMails.QueryRow(mailbox).Scan(&count)
err := t.countMails.QueryRow(accountPubKey, mailbox).Scan(&count)
return count, err
}
func (t *TableMails) MailMove(mailbox string, id int, destination string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.moveMail.Exec(destination, mailbox, id)
func (t *TableMails) MailMove(accountPubKey string, mailbox string, id int, destination string) (int, error) {
var newID int
err := t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
if err := t.selectMailNextID.QueryRow(accountPubKey, destination).Scan(&newID); err != nil {
return err
}
_, err := t.moveMail.Exec(accountPubKey, mailbox, destination, newID, id)
return err
})
return newID, err
}

51
internal/storage/sqlite3/table_queue.go
View File

@ -1,5 +1,6 @@
/*
* Copyright (c) 2021 Neil Alexander
* Copyright (c) 2024 Your Name / Yggmail Contributor
* (Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -27,13 +28,14 @@ type TableQueue struct {
const queueSchema = `
CREATE TABLE IF NOT EXISTS queue (
account_pubkey TEXT NOT NULL,
destination TEXT NOT NULL,
mailbox TEXT NOT NULL,
mailbox TEXT NOT NULL DEFAULT 'Outbox',
id INTEGER NOT NULL,
mail TEXT NOT NULL,
mail TEXT NOT NULL, -- Это поле From в оригинале
rcpt TEXT NOT NULL,
PRIMARY KEY (destination, mailbox, id),
FOREIGN KEY (mailbox, id) REFERENCES mails(mailbox, id) ON DELETE CASCADE ON UPDATE CASCADE
PRIMARY KEY (account_pubkey, destination, mailbox, id),
FOREIGN KEY (account_pubkey, mailbox, id) REFERENCES mails(account_pubkey, mailbox, id) ON DELETE CASCADE ON UPDATE CASCADE
);
`
@ -42,20 +44,20 @@ const queueSelectDestinationsStmt = `
`
const queueSelectIDsForDestinationStmt = `
SELECT id, mail, rcpt FROM queue WHERE destination = $1
SELECT account_pubkey, id, mail, rcpt FROM queue WHERE destination = $1
ORDER BY id DESC
`
const queueInsertDestinationForIDStmt = `
INSERT INTO queue (destination, mailbox, id, mail, rcpt) VALUES($1, $2, $3, $4, $5)
INSERT INTO queue (account_pubkey, destination, mailbox, id, mail, rcpt) VALUES($1, $2, $3, $4, $5, $6)
`
const deleteDestinationForIDStmt = `
DELETE FROM queue WHERE destination = $1 AND mailbox = $2 AND id = $3
DELETE FROM queue WHERE account_pubkey = $1 AND destination = $2 AND mailbox = $3 AND id = $4
`
const queueSelectIsMessagePendingSendStmt = `
SELECT COUNT(*) FROM queue WHERE mailbox = $1 AND id = $2
SELECT COUNT(*) FROM queue WHERE account_pubkey = $1 AND mailbox = $2 AND id = $3
`
func NewTableQueue(db *sql.DB, writer *Writer) (*TableQueue, error) {
@ -98,7 +100,7 @@ func (t *TableQueue) QueueListDestinations() ([]string, error) {
}
return nil, fmt.Errorf("t.queueSelectDestinations.Query: %w", err)
}
defer rows.Close() // nolint:errcheck
defer rows.Close()
var destinations []string
for rows.Next() {
var destination string
@ -116,17 +118,20 @@ func (t *TableQueue) QueueMailIDsForDestination(destination string) ([]types.Que
if err == sql.ErrNoRows {
return nil, nil
}
return nil, fmt.Errorf("t.queueSelectDestinations.Query: %w", err)
return nil, fmt.Errorf("t.queueSelectIDsForDestination.Query: %w", err)
}
defer rows.Close() // nolint:errcheck
defer rows.Close()
var ids []types.QueuedMail
for rows.Next() {
var aPk string
var id int
var from, rcpt string
if err := rows.Scan(&id, &from, &rcpt); err != nil {
if err := rows.Scan(&aPk, &id, &from, &rcpt); err != nil {
return nil, fmt.Errorf("rows.Scan: %w", err)
}
ids = append(ids, types.QueuedMail{
AccountPubKey: aPk,
ID: id,
From: from,
Rcpt: rcpt,
@ -135,29 +140,27 @@ func (t *TableQueue) QueueMailIDsForDestination(destination string) ([]types.Que
return ids, nil
}
func (t *TableQueue) QueueInsertDestinationForID(destination string, id int, from, rcpt string) error {
func (t *TableQueue) QueueInsertDestinationForID(pk string, destination string, id int, from, rcpt string) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.queueInsertDestinationForID.Exec(destination, "Outbox", id, from, rcpt)
_, err := t.queueInsertDestinationForID.Exec(pk, destination, "Outbox", id, from, rcpt)
return err
})
}
func (t *TableQueue) QueueDeleteDestinationForID(destination string, id int) error {
func (t *TableQueue) QueueDeleteDestinationForID(pk string, destination string, id int) error {
return t.writer.Do(t.db, nil, func(txn *sql.Tx) error {
_, err := t.queueDeleteIDForDestination.Exec(destination, "Outbox", id)
_, err := t.queueDeleteIDForDestination.Exec(pk, destination, "Outbox", id)
return err
})
}
func (t *TableQueue) QueueSelectIsMessagePendingSend(mailbox string, id int) (bool, error) {
row := t.queueSelectIsMessagePendingSend.QueryRow(mailbox, id)
if err := row.Err(); err != nil && err != sql.ErrNoRows {
return false, fmt.Errorf("row.Err: %w", err)
} else if err == sql.ErrNoRows {
return false, nil
}
func (t *TableQueue) QueueSelectIsMessagePendingSend(pk string, mailbox string, id int) (bool, error) {
row := t.queueSelectIsMessagePendingSend.QueryRow(pk, mailbox, id)
var count int
if err := row.Scan(&count); err != nil {
if err == sql.ErrNoRows {
return false, nil
}
return false, fmt.Errorf("row.Scan: %w", err)
}
return count > 0, nil

65
internal/storage/storage.go
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2021 Neil Alexander
* (Original Copyright (c) 2021 Neil Alexander)
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -8,7 +8,23 @@
package storage
import "github.com/neilalexander/yggmail/internal/storage/types"
import (
"io"
"github.com/neilalexander/yggmail/internal/storage/types"
)
type BlobStore interface {
StoreMail(aPk string, mailbox string, reader io.Reader) (string, int64, error)
ReadMail(location string) (io.ReadCloser, error)
DeleteMail(location string) error
MoveMail(oldLocation string, newMailbox string) (string, error)
}
type MailStore interface {
Storage
MailCreateRaw(pk string, mailbox string, data io.Reader, location string, size int64) (int, error)
}
type Storage interface {
ConfigGet(key string) (string, error)
@ -16,28 +32,33 @@ type Storage interface {
ConfigSetPassword(password string) error
ConfigTryPassword(password string) (bool, error)
MailboxSelect(mailbox string) (bool, error)
MailNextID(mailbox string) (int, error)
MailIDForSeq(mailbox string, id int) (int, error)
MailUnseen(mailbox string) (int, error)
MailboxList(onlySubscribed bool) ([]string, error)
MailboxCreate(name string) error
MailboxRename(old, new string) error
MailboxDelete(name string) error
MailboxSubscribe(name string, subscribed bool) error
AccountGet(pk string) (*types.Account, error)
AccountCreate(acc *types.Account) error
AccountDelete(pk string) error
AccountList() ([]*types.Account, error)
MailCreate(mailbox string, data []byte) (int, error)
MailSelect(mailbox string, id int) (int, *types.Mail, error)
MailSearch(mailbox string) ([]uint32, error)
MailUpdateFlags(mailbox string, id int, seen, answered, flagged, deleted bool) error
MailDelete(mailbox string, id int) error
MailExpunge(mailbox string) error
MailCount(mailbox string) (int, error)
MailMove(mailbox string, id int, destination string) error
MailboxSelect(pk string, mailbox string) (bool, error)
MailNextID(pk string, mailbox string) (int, error)
MailIDForSeq(pk string, mailbox string, seq int) (int, error)
MailUnseen(pk string, mailbox string) (int, error)
MailboxList(pk string, onlySubscribed bool) ([]string, error)
MailboxCreate(pk string, name string) error
MailboxRename(pk string, old, new string) error
MailboxDelete(pk string, name string) error
MailboxSubscribe(pk string, name string, subscribed bool) error
MailCreate(pk string, mailbox string, r io.Reader, size int) (int, error)
MailSelect(pk string, mailbox string, id int) (int, *types.Mail, error)
MailSearch(pk string, mailbox string) ([]uint32, error)
MailUpdateFlags(pk string, mailbox string, id int, seen, answered, flagged, deleted bool) error
MailDelete(pk string, mailbox string, id int) error
MailExpunge(pk string, mailbox string) error
MailCount(pk string, mailbox string) (int, error)
MailMove(pk string, mailbox string, id int, destination string) (int, error)
QueueListDestinations() ([]string, error)
QueueMailIDsForDestination(destination string) ([]types.QueuedMail, error)
QueueInsertDestinationForID(destination string, id int, from, rcpt string) error
QueueDeleteDestinationForID(destination string, id int) error
QueueSelectIsMessagePendingSend(mailbox string, id int) (bool, error)
QueueInsertDestinationForID(pk string, destination string, id int, from, rcpt string) error
QueueDeleteDestinationForID(pk string, destination string, id int) error
QueueSelectIsMessagePendingSend(pk string, mailbox string, id int) (bool, error)
}

42
internal/storage/types/types.go
View File

@ -8,21 +8,57 @@
package types
import "time"
import (
"io"
"time"
)
type Mail struct {
Mailbox string
ID int
Mail []byte
Body io.ReadCloser // Unified interface for content
MailFile string
Size uint32
Date time.Time
Seen bool
Answered bool
Flagged bool
Deleted bool
RawData []byte // For small blobs
}
func (m *Mail) Len() int { return int(m.Size) }
func (m *Mail) Read(p []byte) (n int, err error) {
if m.Body == nil {
return 0, io.EOF
}
return m.Body.Read(p)
}
func (m *Mail) Close() error {
if m.Body == nil {
return nil
}
return m.Body.Close()
}
type QueuedMail struct {
AccountPubKey string
ID int
From string
Rcpt string
}
// Account is represintation of Account on Domain
//
// # Account can be added in host if added Pubkey of Account
//
// Work in client mode if we have Password and Privkey.
// Else we work in relay mode and wait client
type Account struct {
PublicKey string
Aliases string // In Future realise simple-name auth by username
PasswdHash string // Optional: For IMAP client
Privatekey string // Optional: Encrypted by Password for client mode
}

57
internal/utils/address.go
View File

@ -1,5 +1,6 @@
/*
* Copyright (c) 2021 Neil Alexander
* Copyright (c) 2026 Kaiy Ragur
* Original Copyright (c) 2021 Neil Alexander
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
@ -10,33 +11,53 @@ package utils
import (
"crypto/ed25519"
"encoding/hex"
"encoding/base32"
"fmt"
"strings"
)
const Domain = "yggmail"
func CreateAddress(pk ed25519.PublicKey) string {
func CreateAddress(aPk, dPk ed25519.PublicKey) string {
return fmt.Sprintf(
"%s@%s",
pk, Domain,
EncodeString(aPk), EncodeString(dPk),
)
}
func ParseAddress(email string) (ed25519.PublicKey, error) {
at := strings.LastIndex(email, "@")
if at == 0 {
return nil, fmt.Errorf("invalid email address")
}
if email[at+1:] != Domain {
return nil, fmt.Errorf("invalid email domain")
}
pk, err := hex.DecodeString(email[:at])
func EncodeString(k []byte) string {
return strings.ToLower(base32.HexEncoding.WithPadding(base32.NoPadding).EncodeToString(k))
}
func DecodeString(s string) ([]byte, error) {
pk, err := base32.HexEncoding.WithPadding(base32.NoPadding).DecodeString(strings.ToUpper(s))
if err != nil {
return nil, fmt.Errorf("hex.DecodeString: %w", err)
}
ed := make(ed25519.PublicKey, ed25519.PublicKeySize)
copy(ed, pk)
return ed, nil
if len(pk) == 0 {
return nil, fmt.Errorf("utils.DecodeStrings: len %d", len(pk))
}
return pk, nil
}
func ParseAddress(email string) (ed25519.PublicKey, ed25519.PublicKey, error) {
at := strings.LastIndex(email, "@")
if at == 0 {
return nil, nil, fmt.Errorf("invalid email address")
}
// Domain key
host := email[at+1:]
hostPk, err := DecodeString(host)
if err != nil {
return nil, nil, fmt.Errorf("invalid email domain: %w", err)
} else if len(hostPk) != ed25519.PublicKeySize {
return nil, nil, fmt.Errorf("invalid email domain: invalid keysize")
}
// Account key
user := email[:at]
userPk, err := DecodeString(user)
if err != nil {
return nil, nil, fmt.Errorf("invalid email account: %w", err)
} else if len(userPk) != ed25519.PublicKeySize {
return nil, nil, fmt.Errorf("invalid email account: invalid keysize")
}
return hostPk, userPk, nil
}

63
internal/utils/age/age.go Normal file
View File

@ -0,0 +1,63 @@
package age
import (
"crypto/ed25519"
"io"
"filippo.io/age"
"filippo.io/age/armor"
"github.com/neilalexander/yggmail/internal/utils"
"github.com/neilalexander/yggmail/internal/utils/age/bech32"
)
func Recipient(pub ...ed25519.PublicKey) (rcpt []age.Recipient, err error) {
for _, p := range pub {
x, _ := utils.PubEd25519ToX25519(p)
s, _ := bech32.Encode("age", x)
r, err := age.ParseX25519Recipient(s)
if err != nil {
return nil, err
}
rcpt = append(rcpt, r)
}
return rcpt, nil
}
func Identity(pub ed25519.PrivateKey) (age.Identity, error) {
p, _ := utils.PrivEd25519ToX25519(pub)
s, _ := bech32.Encode("AGE-SECRET-KEY-", p)
return age.ParseX25519Identity(s)
}
type armorCloseWrapper struct {
io.WriteCloser // age writer
armorWriter io.WriteCloser
}
func (w *armorCloseWrapper) Close() error {
if err := w.WriteCloser.Close(); err != nil {
return err
}
return w.armorWriter.Close()
}
func EncryptArmor(w io.Writer, i ...age.Recipient) (io.WriteCloser, error) {
aw := armor.NewWriter(w)
ew, err := age.Encrypt(aw, i...)
if err != nil {
aw.Close()
return nil, err
}
return &armorCloseWrapper{ew, aw}, nil
}
func DecryptArmor(r io.Reader, i ...age.Identity) (io.Reader, error) {
r = armor.NewReader(r)
return Decrypt(r, i...)
}
func Encrypt(w io.Writer, i ...age.Recipient) (io.WriteCloser, error) {
return age.Encrypt(w, i...)
}
func Decrypt(r io.Reader, i ...age.Identity) (io.Reader, error) {
return age.Decrypt(r, i...)
}

173
internal/utils/age/bech32/bech32.go Normal file
View File

@ -0,0 +1,173 @@
// Copyright (c) 2017 Takatoshi Nakagawa
// Copyright (c) 2019 The age Authors
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Package bech32 is a modified version of the reference implementation of BIP173.
package bech32
import (
"fmt"
"strings"
)
var charset = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"
var generator = []uint32{0x3b6a57b2, 0x26508e6d, 0x1ea119fa, 0x3d4233dd, 0x2a1462b3}
func polymod(values []byte) uint32 {
chk := uint32(1)
for _, v := range values {
top := chk >> 25
chk = (chk & 0x1ffffff) << 5
chk = chk ^ uint32(v)
for i := range 5 {
bit := top >> i & 1
if bit == 1 {
chk ^= generator[i]
}
}
}
return chk
}
func hrpExpand(hrp string) []byte {
h := []byte(strings.ToLower(hrp))
var ret []byte
for _, c := range h {
ret = append(ret, c>>5)
}
ret = append(ret, 0)
for _, c := range h {
ret = append(ret, c&31)
}
return ret
}
func verifyChecksum(hrp string, data []byte) bool {
return polymod(append(hrpExpand(hrp), data...)) == 1
}
func createChecksum(hrp string, data []byte) []byte {
values := append(hrpExpand(hrp), data...)
values = append(values, []byte{0, 0, 0, 0, 0, 0}...)
mod := polymod(values) ^ 1
ret := make([]byte, 6)
for p := range ret {
shift := 5 * (5 - p)
ret[p] = byte(mod>>shift) & 31
}
return ret
}
func convertBits(data []byte, frombits, tobits byte, pad bool) ([]byte, error) {
var ret []byte
acc := uint32(0)
bits := byte(0)
maxv := byte(1<<tobits - 1)
for idx, value := range data {
if value>>frombits != 0 {
return nil, fmt.Errorf("invalid data range: data[%d]=%d (frombits=%d)", idx, value, frombits)
}
acc = acc<<frombits | uint32(value)
bits += frombits
for bits >= tobits {
bits -= tobits
ret = append(ret, byte(acc>>bits)&maxv)
}
}
if pad {
if bits > 0 {
ret = append(ret, byte(acc<<(tobits-bits))&maxv)
}
} else if bits >= frombits {
return nil, fmt.Errorf("illegal zero padding")
} else if byte(acc<<(tobits-bits))&maxv != 0 {
return nil, fmt.Errorf("non-zero padding")
}
return ret, nil
}
// Encode encodes the HRP and a bytes slice to Bech32. If the HRP is uppercase,
// the output will be uppercase.
func Encode(hrp string, data []byte) (string, error) {
values, err := convertBits(data, 8, 5, true)
if err != nil {
return "", err
}
if len(hrp) < 1 {
return "", fmt.Errorf("invalid HRP: %q", hrp)
}
for p, c := range hrp {
if c < 33 || c > 126 {
return "", fmt.Errorf("invalid HRP character: hrp[%d]=%d", p, c)
}
}
if strings.ToUpper(hrp) != hrp && strings.ToLower(hrp) != hrp {
return "", fmt.Errorf("mixed case HRP: %q", hrp)
}
lower := strings.ToLower(hrp) == hrp
hrp = strings.ToLower(hrp)
var ret strings.Builder
ret.WriteString(hrp)
ret.WriteString("1")
for _, p := range values {
ret.WriteByte(charset[p])
}
for _, p := range createChecksum(hrp, values) {
ret.WriteByte(charset[p])
}
if lower {
return ret.String(), nil
}
return strings.ToUpper(ret.String()), nil
}
// Decode decodes a Bech32 string. If the string is uppercase, the HRP will be uppercase.
func Decode(s string) (hrp string, data []byte, err error) {
if strings.ToLower(s) != s && strings.ToUpper(s) != s {
return "", nil, fmt.Errorf("mixed case")
}
pos := strings.LastIndex(s, "1")
if pos < 1 || pos+7 > len(s) {
return "", nil, fmt.Errorf("separator '1' at invalid position: pos=%d, len=%d", pos, len(s))
}
hrp = s[:pos]
for p, c := range hrp {
if c < 33 || c > 126 {
return "", nil, fmt.Errorf("invalid character human-readable part: s[%d]=%d", p, c)
}
}
s = strings.ToLower(s)
for p, c := range s[pos+1:] {
d := strings.IndexRune(charset, c)
if d == -1 {
return "", nil, fmt.Errorf("invalid character data part: s[%d]=%v", p, c)
}
data = append(data, byte(d))
}
if !verifyChecksum(hrp, data) {
return "", nil, fmt.Errorf("invalid checksum")
}
data, err = convertBits(data[:len(data)-6], 5, 8, false)
if err != nil {
return "", nil, err
}
return hrp, data, nil
}

92
internal/utils/crypt.go Normal file
View File

@ -0,0 +1,92 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package utils
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/base64"
"fmt"
"io"
"golang.org/x/crypto/scrypt"
)
// Get base64 string of encrpted data
// salt [:16]
// nonce [16:28]
// data [28:]
func AESEncrypt(rawKey string, password string) (string, error) {
salt := make([]byte, 16)
if _, err := io.ReadFull(rand.Reader, salt); err != nil {
return "", err
}
key, err := scrypt.Key([]byte(password), salt, 32768, 8, 1, 32)
if err != nil {
return "", err
}
block, err := aes.NewCipher(key)
if err != nil {
return "", err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
nonce := make([]byte, gcm.NonceSize())
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
return "", err
}
ciphertext := gcm.Seal(nil, nonce, []byte(rawKey), nil)
result := append(salt, append(nonce, ciphertext...)...)
return base64.StdEncoding.EncodeToString(result), nil
}
func AESDecrypt(encryptedB64 string, password string) (string, error) {
data, err := base64.StdEncoding.DecodeString(encryptedB64)
if err != nil {
return "", err
}
if len(data) < 16+12 { // Salt(16) + Nonce(12)
return "", fmt.Errorf("invalid encrypted data")
}
salt := data[:16]
nonce := data[16:28]
ciphertext := data[28:]
key, err := scrypt.Key([]byte(password), salt, 32768, 8, 1, 32)
if err != nil {
return "", err
}
block, err := aes.NewCipher(key)
if err != nil {
return "", err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
rawKey, err := gcm.Open(nil, nonce, ciphertext, nil)
if err != nil {
return "", fmt.Errorf("decryption failed (wrong password?): %w", err)
}
return string(rawKey), nil
}

80
internal/utils/e2ee/crypt.go Normal file
View File

@ -0,0 +1,80 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package e2ee
import (
"crypto/ed25519"
"crypto/sha512"
"io"
"os"
)
type Encryptor func(io.Writer, ...ed25519.PublicKey) (io.WriteCloser, error)
type Decryptor func(io.Reader) (io.Reader, error)
type Verifier func([]byte, []byte) error
type Signer func([]byte) ([]byte, error)
func EncryptPack(input io.ReadSeeker, rcpt []ed25519.PublicKey, signer Signer, enc Encryptor) (io.Reader, error) {
input.Seek(0, io.SeekStart)
h := sha512.New()
if _, err := io.Copy(h, input); err != nil {
return nil, err
}
sig, err := signer(h.Sum(nil))
if err != nil {
return nil, err
}
pr, pw := io.Pipe()
go func() {
input.Seek(0, io.SeekStart)
ew, _ := enc(pw, rcpt...)
ew.Write(sig)
io.Copy(ew, input)
ew.Close()
pw.Close()
}()
return pr, nil
}
func DecryptPack(input io.Reader, dec Decryptor, verifier Verifier) (io.ReadCloser, error) {
dr, err := dec(input)
if err != nil {
return nil, err
}
sig := make([]byte, 64)
if _, err := io.ReadFull(dr, sig); err != nil {
return nil, err
}
tmp, _ := os.CreateTemp("", "ygg-mail-*")
h := sha512.New()
io.Copy(io.MultiWriter(tmp, h), dr)
if err := verifier(h.Sum(nil), sig); err != nil {
tmp.Close()
os.Remove(tmp.Name())
return nil, err
}
tmp.Seek(0, io.SeekStart)
return &tempFileCloser{File: tmp}, nil
}
type tempFileCloser struct {
*os.File
}
func (t *tempFileCloser) Close() error {
t.File.Close()
return os.Remove(t.Name())
}

83
internal/utils/e2ee/e2ee_test.go Normal file
View File

@ -0,0 +1,83 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package e2ee_test
import (
"bytes"
"crypto/ed25519"
"io"
"testing"
"github.com/neilalexander/yggmail/internal/utils"
"github.com/neilalexander/yggmail/internal/utils/age"
"github.com/neilalexander/yggmail/internal/utils/e2ee"
)
func TestFullE2EECycle(t *testing.T) {
_, senderPriv, _ := ed25519.GenerateKey(nil)
senderPub := senderPriv.Public().(ed25519.PublicKey)
_, recipientPriv, _ := ed25519.GenerateKey(nil)
recipientPub := recipientPriv.Public().(ed25519.PublicKey)
_, domainPriv, _ := ed25519.GenerateKey(nil)
domainPub := domainPriv.Public().(ed25519.PublicKey)
senderAddr := utils.CreateAddress(senderPub, domainPub) // например: userkey@domainkey
original := []byte("From: " + senderAddr + "\r\nSubject: test\r\n\r\nHello world! Это тест.")
encryptedReader, err := e2ee.EncryptFilter(
bytes.NewReader(original),
senderAddr,
[]ed25519.PublicKey{recipientPub},
func(data []byte) ([]byte, error) {
return ed25519.Sign(senderPriv, data), nil
},
func(w io.Writer, rcpt ...ed25519.PublicKey) (io.WriteCloser, error) {
r, err := age.Recipient(rcpt...)
if err != nil {
return nil, err
}
return age.EncryptArmor(w, r...)
},
)
if err != nil {
t.Fatal("EncryptFilter failed:", err)
}
encryptedBytes, _ := io.ReadAll(encryptedReader)
encryptedReader.Close()
stored := bytes.NewReader(encryptedBytes)
decryptor := func(r io.Reader) (io.Reader, error) {
id, err := age.Identity(recipientPriv)
if err != nil {
return nil, err
}
return age.DecryptArmor(r, id)
}
decrypted, err := e2ee.DecryptFilter(stored, decryptor)
if err != nil {
t.Fatal("DecryptFilter failed:", err)
}
result, _ := io.ReadAll(decrypted)
decrypted.Close()
if !bytes.Contains(result, []byte("Hello world! Это тест.")) {
t.Fatalf("Decryption failed, got:\n%s", string(result))
}
if !bytes.Contains(result, []byte("Subject: test")) {
t.Fatalf("Original headers lost, got:\n%s", string(result))
}
t.Log("Full E2EE cycle OK. Decrypted body preview:\n", string(result[:200]))
}

101
internal/utils/e2ee/msg.go Normal file
View File

@ -0,0 +1,101 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package e2ee
import (
"crypto/ed25519"
"fmt"
"io"
"net/mail"
"time"
"github.com/emersion/go-message"
"github.com/neilalexander/yggmail/internal/utils"
)
func EncryptFilter(
origMsg io.ReadSeeker,
fromAddr string,
rcptPk []ed25519.PublicKey,
mySign Signer,
myEnc Encryptor,
) (io.ReadCloser, error) {
packReader, err := EncryptPack(origMsg, rcptPk, mySign, myEnc)
if err != nil {
return nil, fmt.Errorf("failed to sign and encrypt: %w", err)
}
hdr := new(message.Header)
hdr.Set("From", fromAddr)
hdr.Set("Date", time.Now().Format(time.RFC1123Z))
hdr.Set("X-Yggmail-Encryption", "v1")
pr, pw := io.Pipe()
go func() {
tw, err := message.CreateWriter(pw, *hdr)
if err != nil {
pw.CloseWithError(err)
return
}
_, err = io.Copy(tw, packReader)
tw.Close()
pw.CloseWithError(err)
}()
return pr, nil
}
func DecryptFilter(r io.ReadSeeker, decryptor Decryptor) (io.ReadCloser, error) {
r.Seek(0, io.SeekStart)
msg, err := mail.ReadMessage(r)
if err != nil {
r.Seek(0, io.SeekStart)
return io.NopCloser(r), nil
}
if msg.Header.Get("X-Yggmail-Encryption") != "v1" {
r.Seek(0, io.SeekStart)
return io.NopCloser(r), nil
}
fromStr := msg.Header.Get("From")
fPk := extractPublicKey(fromStr)
if fPk == nil {
r.Seek(0, io.SeekStart)
return io.NopCloser(r), nil
}
innerVerify := func(data []byte, sig []byte) error {
if !ed25519.Verify(fPk, data, sig) {
return fmt.Errorf("inner signature invalid")
}
return nil
}
decrypted, err := DecryptPack(msg.Body, decryptor, innerVerify)
if err != nil {
r.Seek(0, io.SeekStart)
return nil, fmt.Errorf("decrypt enc packet error: %w", err)
}
return decrypted, nil
}
func extractPublicKey(addr string) ed25519.PublicKey {
a, err := mail.ParseAddress(addr)
if err != nil {
return nil
}
addr = a.Address
_, aPk, err := utils.ParseAddress(addr)
if err != nil {
return nil
}
return aPk
}

32
internal/utils/x25519.go Normal file
View File

@ -0,0 +1,32 @@
/*
* Copyright (c) 2026 Kaiy Ragur
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package utils
import (
"crypto/ed25519"
"crypto/sha512"
"filippo.io/edwards25519"
)
func PubEd25519ToX25519(pub ed25519.PublicKey) ([]byte, error) {
point, err := edwards25519.NewIdentityPoint().SetBytes(pub)
if err != nil {
return nil, err
}
x25519Pub := point.BytesMontgomery()
return x25519Pub, nil
}
func PrivEd25519ToX25519(priv ed25519.PrivateKey) ([]byte, error) {
seed := priv.Seed()
h := sha512.Sum512(seed)
x25519Priv := h[:32]
return x25519Priv, nil
}

7
internal/welcome/welcome.go
View File

@ -3,9 +3,10 @@ package welcome
import (
"bytes"
"fmt"
"log"
"github.com/emersion/go-message"
"github.com/neilalexander/yggmail/internal/storage"
"log"
)
const (
@ -27,14 +28,14 @@ func Onboard(user string, storage storage.Storage, log *log.Logger) {
log.Println("Failure to generate welcome message")
}
var welcomeId int
if id, e := storage.MailCreate("INBOX", welcomeMsg); e != nil {
if id, e := storage.MailCreate(user, "INBOX", bytes.NewBuffer(welcomeMsg), len(welcomeMsg)); e != nil {
log.Printf("Failed to store welcome message: %v\n", e)
panic("See above")
} else {
welcomeId = id
}
if storage.MailUpdateFlags("INBOX", welcomeId, false, false, false, false) != nil {
if storage.MailUpdateFlags(user, "INBOX", welcomeId, false, false, false, false) != nil {
panic("Could not set flags on onboarding message")
}