Add feed generation

The ODDMU_WEBFINGER environment variable can be used to switch it on,
but it doesn't work on the command line.

Makefile

@@ -10,11 +10,17 @@ help:
 	@echo make test
 	@echo "    runs the tests"
 	@echo
-	@echo make upload
-	@echo "    this is how I upgrade my server"
+	@echo make docs
+	@echo "    create man pages from text files"
 	@echo
 	@echo go build
 	@echo "    just build it"
+	@echo
+	@echo make install
+	@echo "    install the files to ~/.local"
+	@echo
+	@echo make upload
+	@echo "    this is how I upgrade my server"
 
 run:
 	go run .
@@ -24,5 +30,15 @@ test:
 
 upload:
 	go build
-	rsync --itemize-changes --archive oddmu oddmu.service *.html README.md sibirocobombus.root:/home/oddmu/
+	rsync --itemize-changes --archive oddmu sibirocobombus.root:/home/oddmu/
 	ssh sibirocobombus.root "systemctl restart oddmu; systemctl restart alex"
+	@echo Changes to the template files need careful consideration
+
+docs:
+	cd man; make
+
+install:
+	make docs
+	for n in 1 5 7; do install -D -t $$HOME/.local/share/man/man$$n man/*.$$n; done
+	go build
+	install -D -t $$HOME/.local/bin oddmu

README.md

@@ -13,151 +13,17 @@ repository that results from these discussions.
 The wiki lists no recent changes. The expectation is that the people
 that care were involved in the discussions beforehand.
 
-The wiki also produces no feed. The assumption is that announcements
-are made on social media: blogs, news aggregators, discussion forums,
-the fediverse, but humans. There is no need for bots.
-
-As you'll see below, the idea is that the webserver handles as many
-tasks as possible. It logs requests, does rate limiting, handles
-encryption, gets the certificates, and so on. The web server acts as a
-reverse proxy and the wiki ends up being a content management system
-with almost no structure – or endless malleability, depending on your
-point of view.
-
-And last but not least: µ is the letter mu, so Oddµ is usually written
-Oddmu. 🙃
-
-## Markdown
-
 This wiki uses a [Markdown
 library](https://github.com/gomarkdown/markdown) to generate the web
-pages from Markdown. There is no additional wiki markup. Most
-importantly, double square brackets are not a link. If you're used to
-that, it'll be strange as you need to repeat the name: `[like
-this](like this)`.
+pages from Markdown. There are two extensions Oddmu adds to the
+library: local links `[[like this]]` and hashtags `#Like_This`.
 
-The Markdown processor comes with a few extensions, some of which are
-enable by default:
+This wiki uses the [lingua](github.com/pemistahl/lingua-go) library to
+detect languages in order to get hyphenation right.
 
-* emphasis markers inside words are ignored
-* tables are supported
-* fenced code blocks are supported
-* autolinking of "naked" URLs are supported
-* strikethrough using two tildes is supported (`~~like this~~`)
-* it is strict about prefix heading rules
-* you can specify an id for headings (`{#id}`)
-* trailing backslashes turn into line breaks
-* definition lists are supported
-* MathJax is supported (but needs a separte setup)
-
-A table with footers and a columnspan:
-
-```text
-Name    | Age
---------|------
-Bob     ||
-Alice   | 23
-========|======
-Total   | 23
-```
-
-A definition list:
-
-```text
-Cat
-: Fluffy animal everyone likes
-
-Internet
-: Vector of transmission for pictures of cats
-```
-
-There is another extension made: hashtags link to searches for the
-hashtag. Hashtags are separate from titles because there is no space
-after the hash. Use the underscore to use hashtags consisting of
-multiple words.
-
-```
-# Title
-
-Text
-
-#Tag #Another_Tag
-```
-
-## Templates
-
-The template files are the HTML files in the working directory:
-`add.html`, `edit.html`, `search.html`, `upload.html` and `view.html`.
-Feel free to change the templates and restart the server. The first
-change you should make is to replace the email address in `view.html`.
-😄
-
-See [Structuring the web
-with HTML](https://developer.mozilla.org/en-US/docs/Learn/HTML) to
-learn more about HTML.
-
-Modifying the styles in the templates would be another good start to
-get a feel for it. See [Learn to style HTML using
-CSS](https://developer.mozilla.org/en-US/docs/Learn/CSS) to learn more
-about style sheets.
-
-The templates can refer to the following properties of a page:
-
-`{{.Title}}` is the page title. If the page doesn't provide its own
-title, the page name is used.
-
-`{{.Name}}` is the page name, escaped for use in URLs. More
-specifically, it is URI escaped except for the slashes. The page name
-doesn't include the `.md` extension.
-
-`{{.Html}}` is the rendered Markdown, as HTML.
-
-`{{printf "%s" .Body}}` is the Markdown, as a string (the data itself
-is a byte array and that's why we need to call `printf`).
-
-For the `search.html` template only:
-
-`{{.Results}}` indicates if there were any search results.
-
-`{{.Items}}` is an array of pages, each containing a search result. A
-search result is a page (with the properties seen above). Thus, to
-refer to them, you need to use a `{{range .Items}}` … `{{end}}`
-construct.
-
-For search results, `{{.Html}}` is the rendered Markdown of a page
-summary, as HTML.
-
-`{{.Score}}` is a numerical score for search results.
-
-The `upload.html` template cannot refer to anything.
-
-When calling the `save` action, the page name is take from the URL and
-the page content is taken from the `body` form parameter. To
-illustrate, here's how to edit a page using `curl`:
-
-```sh
-curl --form body="Did you bring a towel?" \
-  http://localhost:8080/save/welcome
-```
-
-The wiki uses the standard
-[html/template](https://pkg.go.dev/html/template) library to do this.
-There's more information on writing templates in the documentation for
-the [text/template](https://pkg.go.dev/text/template) library.
-
-## Non-English hyphenation
-
-Automatic hyphenation by the browser requires two things: The style
-sheet must indicate `hyphen: auto` for an HTML element such as `body`,
-and that element must have a `lang` set (usually a two letter language
-code such as `de` for German). This happens in the template files,
-such as `view.html` and `search.html`.
-
-Oddmu uses the [lingua](github.com/pemistahl/lingua-go) library to
-detect languages. If you know that you're only going to use a small
-number of languages – or just a single language! – you can set the
-environment variable ODDMU_LANGUAGES to a comma-separated list of ISO
-639-1 codes, e.g. "en" or "en,de,fr,pt".
+This wiki uses the standard
+[html/template](https://pkg.go.dev/html/template) library to generate
+HTML.
 
 ## Building
 
@@ -165,7 +31,7 @@ environment variable ODDMU_LANGUAGES to a comma-separated list of ISO
 go build
 ```
 
-## Test
+## Running
 
 The working directory is where pages are saved and where templates are
 loaded from. You need a copy of the template files in this directory.
@@ -176,277 +42,7 @@ go run .
 ```
 
 The program serves the local directory as a wiki on port 8080. Point
-your browser to http://localhost:8080/ to get started. This is
-equivalent to http://localhost:8080/view/index – the first page
-you'll create, most likely.
-
-If you ran it in the source directory, try
-http://localhost:8080/view/README – this serves the README file you're
-currently reading.
-
-You can change the port by setting the ODDMU_PORT environment
-variable.
-
-## Deploying it using systemd
-
-As root, on your server:
-
-```sh
-adduser --system --home /home/oddmu oddmu
-```
-
-Copy all the files into `/home/oddmu` to your server: `oddmu`,
-`oddmu.service`, and all the template files ending in `.html`.
-
-Edit the `oddmu.service` file. These are the three lines you most
-likely have to take care of:
-
-```
-ExecStart=/home/oddmu/oddmu
-WorkingDirectory=/home/oddmu
-Environment="ODDMU_PORT=8080"
-Environment="ODDMU_LANGUAGES=en,de"
-```
-
-Install the service file and enable it:
-
-```sh
-ln -s /home/oddmu/oddmu.service /etc/systemd/system/
-systemctl enable --now oddmu
-```
-
-Check the log:
-
-```sh
-journalctl --unit oddmu
-```
-
-Follow the log:
-
-```sh
-journalctl --follow --unit oddmu
-```
-
-Edit the first page using `lynx`:
-
-```sh
-lynx http://localhost:8080/view/index
-```
-
-## Web server setup
-
-HTTPS is not part of the wiki. You probably want to configure this in
-your webserver. I guess you could use stunnel, too. If you're using
-Apache, you might have set up a site like I did, below. In my case,
-that'd be `/etc/apache2/sites-enabled/500-transjovian.conf`:
-
-```apache
-MDomain transjovian.org
-MDCertificateAgreement accepted
-
-<VirtualHost *:80>
-    ServerName transjovian.org
-    RewriteEngine on
-    RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [redirect]
-</VirtualHost>
-<VirtualHost *:443>
-    ServerAdmin alex@alexschroeder.ch
-    ServerName transjovian.org
-    SSLEngine on
-    ProxyPassMatch ^/(search|(view|edit|save|add|append|upload|drop)/(.*))?$ http://localhost:8080/$1
-</VirtualHost>
-```
-
-First, it manages the domain, getting the necessary certificates. It
-redirects regular HTTP traffic from port 80 to port 443. It turns on
-the SSL engine for port 443. It proxies the requests for the wiki to
-port 8080.
-
-Thus, this is what happens:
-
-* The user tells the browser to visit `transjovian.org`
-* The browser sends a request for `http://transjovian.org` (on port 80)
-* Apache redirects this to `https://transjovian.org/` by default (now on port 443)
-* This is proxied to `http://transjovian.org:8080/` (no encryption, on port 8080)
-
-Restart the server, gracefully:
-
-```
-apachectl graceful
-```
-
-To serve both HTTP and HTTPS, don't redirect from the first virtual
-host to the second – instead just proxy to the wiki like you did for
-the second virtual host: use a copy of the `ProxyPassMatch` directive
-instead of `RewriteEngine on` and `RewriteRule`.
-
-## Access
-
-Access control is not part of the wiki. By default, the wiki is
-editable by all. This is most likely not what you want unless you're
-running it stand-alone, unconnected to the Internet.
-
-You probably want to configure this in your webserver. If you're using
-Apache, you might have set up a site like the following.
-
-Create a new password file called `.htpasswd` and add the user "alex":
-
-```sh
-cd /home/oddmu
-htpasswd -c .htpasswd alex
-```
-
-To add more users, don't use the `-c` option or you will overwrite it!
-
-To add another user:
-
-```sh
-htpasswd .htpasswd berta
-```
-
-To delete remove a user:
-
-```sh
-htpasswd -D .htpasswd berta
-```
-
-Modify your site configuration and protect the `/edit/`, `/save/`,
-`/add/`, `/append/`, `/upload/` and `/drop/` URLs with a password by
-adding the following to your `<VirtualHost *:443>` section:
-
-```apache
-<LocationMatch "^/(edit|save|add|append|upload|drop)/">
-  AuthType Basic
-  AuthName "Password Required"
-  AuthUserFile /home/oddmu/.htpasswd
-  Require valid-user
-</LocationMatch>
-```
-
-## Serve static files
-
-If you want to serve static files as well, add a document root to your
-webserver configuration. Using Apache, for example:
-
-```apache
-DocumentRoot /home/oddmu/static
-<Directory /home/oddmu/static>
-    Require all granted
-</Directory>
-```
-
-Create this directory, making sure to give it a permission that your
-webserver can read (world readable file, world readable and executable
-directory). Populate it with files.
-
-Make sure that none of the static files look like the wiki paths
-`/view/`, `/edit/`, `/save/`, `/add/`, `/append/`, `/upload/`, `/drop/`
-or `/search`. For example, create a file called `robots.txt`
-containing the following, tellin all robots that they're not welcome.
-
-```text
-User-agent: *
-Disallow: /
-```
-
-You site now serves `/robots.txt` without interfering with the wiki,
-and without needing a wiki page.
-
-[Wikipedia](https://en.wikipedia.org/wiki/Robot_exclusion_standard)
-has more information.
-
-## Different logins for different access rights
-
-What if you have a site with various subdirectories and each
-subdirectory is for a different group of friends? You can set this up
-using your webserver. One way to do this is to require specific
-usernames (which must have a password in the password file mentioned
-above.
-
-This requires a valid login by the user "alex" or "berta":
-
-```apache
-<LocationMatch "^/(edit|save|add|append|upload|drop)/intetebi/">
-  Require user alex berta
-</LocationMatch>
-```
-
-## Private wikis
-
-Based on the above, you can prevent people from reading the wiki, too.
-The `LocationMatch` must cover the `/view/` URLs. In order to protect
-*everything*, use a [Location directive](https://httpd.apache.org/docs/current/mod/core.html#location)
-that matches everything:
-
-```apache
-<Location />
-  AuthType Basic
-  AuthName "Password Required"
-  AuthUserFile /home/oddmu/.htpasswd
-  Require valid-user
-</Location>
-```
-
-## Virtual hosting
-
-Virtual hosting in this context means that the program serves two
-different sites for two different domains from the same machine. Oddmu
-doesn't support that, but your webserver does. Therefore, start an
-Oddmu instance for every domain name, each listening on a different
-port. Then set up your web server such that ever domain acts as a
-reverse proxy to a different Oddmu instance.
-
-## Understanding search
-
-The index indexes trigrams. Each group of three characters is a
-trigram. A document with content "This is a test" is turned to lower
-case and indexed under the trigrams "thi", "his", "is ", "s i", " is",
-"is ", "s a", " a ", "a t", " te", "tes", "est".
-
-Each query is split into words and then processed the same way. A
-query with the words "this test" is turned to lower case and produces
-the trigrams "thi", "his", "tes", "est". This means that the word
-order is not considered when searching for documents.
-
-This also means that there is no stemming. Searching for "testing"
-won't find "This is a test" because there are no matches for the
-trigrams "sti", "tin", "ing".
-
-These trigrams are looked up in the index, resulting in the list of
-documents. Each document found is then scored. Each of the following
-increases the score by one point:
-
-- the entire phrase matches
-- a word matches
-- a word matches at the beginning of a word
-- a word matches at the end of a word
-- a word matches as a whole word
-
-A document with content "This is a test" when searched with the phrase
-"this test" therefore gets a score of 8: the entire phrase does not
-match but each word gets four points.
-
-Trigrams are sometimes strange: In a text containing the words "main"
-and "rail", a search for "mail" returns a match because the trigrams
-"mai" and "ail" are found. In this situation, the result has a score
-of 0.
-
-## Limitations
-
-Page titles are filenames with `.md` appended. If your filesystem
-cannot handle it, it can't be a page name.
-
-The pages are indexed as the server starts and the index is kept in
-memory. If you have a ton of pages, this surely wastes a lot of
-memory.
-
-Files may not end with a tilde (`~`) – these are backup files.
-
-You cannot edit uploaded files. If you upload a file called
-`hello.txt` and attempt to edit it by using `/edit/hello.txt` you will
-create a page with the name `hello.txt.md` instead.
-
-You cannot delete uploaded files via the web.
+your browser to http://localhost:8080/ to use it.
 
 ## Bugs
 
@@ -456,9 +52,3 @@ If you spot any, [contact](https://alexschroeder.ch/wiki/Contact) me.
 
 [Writing Web Applications](https://golang.org/doc/articles/wiki/)
 provided the initial code for this wiki.
-
-For the proxy stuff, see
-[Apache: mod_proxy](https://httpd.apache.org/docs/current/mod/mod_proxy.html).
-
-For the usernames and password stuff, see
-[Apache: Authentication and Authorization](https://httpd.apache.org/docs/current/howto/auth.html).

accounts.go

@@ -0,0 +1,161 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"github.com/gomarkdown/markdown/ast"
+	"github.com/gomarkdown/markdown/parser"
+	"io"
+	"net/http"
+	"os"
+	"sync"
+)
+
+// useWebfinger indicates whether Oddmu looks up the profile pages of
+// fediverse accounts. To enable this, set the environment variable
+// ODDMU_WEBFINGER to "1".
+var useWebfinger = false
+
+// Accounts contains the map used to set the usernames. Make sure to
+// lock and unlock as appropriate.
+type Accounts struct {
+	sync.RWMutex
+
+	// uris is a map, mapping account names likes
+	// "@alex@alexschroeder.ch" to URIs like
+	// "https://social.alexschroeder.ch/@alex".
+	uris map[string]string
+}
+
+// accounts holds the global mapping of accounts to profile URIs.
+var accounts Accounts
+
+// initAccounts sets up the accounts map. This is called once at
+// startup and therefore does not need to be locked. On ever restart,
+// this map starts empty and is slowly repopulated as pages are
+// visited.
+func initAccounts() {
+	if os.Getenv("ODDMU_WEBFINGER") == "1" {
+		accounts.uris = make(map[string]string)
+		useWebfinger = true
+	}
+}
+
+// account links a social media account like @account@domain to a
+// profile page like https://domain/user/account. Any account seen for
+// the first time uses a best guess profile URI. It is also looked up
+// using webfinger, in parallel. See lookUpAccountUri. If the lookup
+// succeeds, the best guess is replaced with the new URI so on
+// subsequent requests, the URI is correct.
+func account(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
+	data = data[offset:]
+	i := 1 // skip @ of username
+	n := len(data)
+	d := 0
+	for i < n && (data[i] >= 'a' && data[i] <= 'z' ||
+		data[i] >= 'A' && data[i] <= 'Z' ||
+		data[i] >= '0' && data[i] <= '9' ||
+		data[i] == '@' ||
+		data[i] == '.' ||
+		data[i] == '-') {
+		if data[i] == '@' {
+			if d != 0 {
+				// more than one @ is invalid
+				return 0, nil
+			} else {
+				d = i + 1 // skip @ of domain
+			}
+		}
+		i++
+	}
+	for i > 1 && (data[i-1] == '.' ||
+		data[i-1] == '-') {
+		i--
+	}
+	if i == 0 || d == 0 {
+		return 0, nil
+	}
+	user := data[0 : d-1] // includes @
+	domain := data[d:i]   // excludes @
+	account := data[1:i]  // excludes @
+	accounts.RLock()
+	uri, ok := accounts.uris[string(account)]
+	defer accounts.RUnlock()
+	if !ok {
+		fmt.Printf("Looking up %s\n", account)
+		uri = "https://" + string(domain) + "/users/" + string(user[1:])
+		accounts.uris[string(account)] = uri // prevent more lookings
+		go lookUpAccountUri(string(account), string(domain))
+	}
+	link := &ast.Link{
+		Destination: []byte(uri),
+		Title:       data[0:i],
+	}
+	ast.AppendChild(link, &ast.Text{Leaf: ast.Leaf{Literal: data[0 : d-1]}})
+	return i, link
+}
+
+// lookUpAccountUri is called for accounts that haven't been seen
+// before. It calls webfinger and parses the JSON. If possible, it
+// extracts the link to the profile page and replaces the entry in
+// accounts.
+func lookUpAccountUri(account, domain string) {
+	uri := "https://" + domain + "/.well-known/webfinger"
+	resp, err := http.Get(uri + "?resource=acct:" + account)
+	if err != nil {
+		fmt.Printf("Failed to look up %s: %s\n", account, err)
+		return
+	}
+	defer resp.Body.Close()
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		fmt.Printf("Failed to read from %s: %s\n", account, err)
+		return
+	}
+	var wf WebFinger
+	err = json.Unmarshal([]byte(body), &wf)
+	if err != nil {
+		fmt.Printf("Failed to parse the JSON from %s: %s\n", account, err)
+		return
+	}
+	uri, err = parseWebFinger(body)
+	if err != nil {
+		fmt.Printf("Could not find profile URI for %s: %s\n", account, err)
+	}
+	fmt.Printf("Found profile for %s: %s\n", account, uri)
+	accounts.Lock()
+	defer accounts.Unlock()
+	accounts.uris[account] = uri
+}
+
+// Link a link in the WebFinger JSON.
+type Link struct {
+	Rel  string `json:"rel"`
+	Type string `json:"type"`
+	Href string `json:"href"`
+}
+
+// WebFinger is a structure used to unmarshall JSON.
+type WebFinger struct {
+	Subject string   `json:"subject"`
+	Aliases []string `json:"aliases"`
+	Links   []Link   `json:"links"`
+}
+
+// parseWebFinger parses the web finger JSON and returns the profile
+// page URI. For unmarshalling the JSON, it uses the Link and
+// WebFinger structs.
+func parseWebFinger(body []byte) (string, error) {
+	var wf WebFinger
+	err := json.Unmarshal(body, &wf)
+	if err != nil {
+		return "", err
+	}
+	for _, link := range wf.Links {
+		if link.Rel == "http://webfinger.net/rel/profile-page" &&
+			link.Type == "text/html" {
+			return link.Href, nil
+		}
+	}
+	return "", err
+}

accounts_test.go

@@ -0,0 +1,45 @@
+package main
+
+import (
+	"github.com/stretchr/testify/assert"
+	"testing"
+)
+
+// This causes network access!
+// func TestPageAccount(t *testing.T) {
+// 	initAccounts()
+// 	p := &Page{Body: []byte(`@alex, @alex@alexschroeder.ch said`)}
+// 	p.renderHtml()
+// 	r := `<p>@alex, <a href="https://alexschroeder.ch/users/alex" rel="nofollow">@alex</a> said</p>
+// `
+// 	assert.Equal(t, r, string(p.Html))
+// }
+
+func TestWebfingerParsing(t *testing.T) {
+	body := []byte(`{
+  "subject": "acct:Gargron@mastodon.social",
+  "aliases": [
+    "https://mastodon.social/@Gargron",
+    "https://mastodon.social/users/Gargron"
+  ],
+  "links": [
+    {
+      "rel": "http://webfinger.net/rel/profile-page",
+      "type": "text/html",
+      "href": "https://mastodon.social/@Gargron"
+    },
+    {
+      "rel": "self",
+      "type": "application/activity+json",
+      "href": "https://mastodon.social/users/Gargron"
+    },
+    {
+      "rel": "http://ostatus.org/schema/1.0/subscribe",
+      "template": "https://mastodon.social/authorize_interaction?uri={uri}"
+    }
+  ]
+}`)
+	uri, err := parseWebFinger(body)
+	assert.NoError(t, err)
+	assert.Equal(t, "https://mastodon.social/@Gargron", uri)
+}

add.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
     <meta charset="utf-8">
     <meta name="format-detection" content="telephone=no">
@@ -13,7 +13,7 @@ form, textarea { width: 100%; }
   <body>
     <h1>Adding to {{.Title}}</h1>
     <form action="/append/{{.Name}}" method="POST">
-      <textarea name="body" rows="20" cols="80" placeholder="Text" autofocus required></textarea>
+      <textarea name="body" rows="20" cols="80" placeholder="Text" lang="" autofocus required></textarea>
       <p><input type="submit" value="Add">
         <a href="/view/{{.Name}}"><button type="button">Cancel</button></a></p>
     </form>

add_append_test.go

@@ -12,7 +12,7 @@ import (
 func TestAddAppend(t *testing.T) {
 	_ = os.RemoveAll("testdata")
 	index.load()
-	
+
 	p := &Page{Name: "testdata/fire", Body: []byte(`# Fire
 Orange sky above
 Reflects a distant fire

commands.go

@@ -1,36 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"os"
-)
-
-func commands() {
-	if len(os.Args) == 3 && os.Args[1] == "html" {
-		p, err := loadPage(os.Args[2]);
-		if err != nil {
-			fmt.Println(err);
-		} else {
-			p.renderHtml();
-			fmt.Println(p.Html);
-		}
-	} else if len(os.Args) > 2 && os.Args[1] == "search" {
-		index.load()
-		for _, q := range os.Args[2:] {
-			items := search(q)
-			fmt.Printf("Search %s: %d results\n", q, len(items))
-			for _, p := range items {
-				fmt.Printf("* %s (%d)\n", p.Title, p.Score)
-			}
-		}
-	} else {
-		fmt.Printf("Unknown command: %v\n", os.Args[1:])
-		fmt.Print("Without any arguments, serves a wiki.\n")
-		fmt.Print("  Environment variable ODDMUSE_PORT controls the port.\n")
-		fmt.Print("  Environment variable ODDMUSE_LANGAUGES controls the languages detected.\n")
-		fmt.Print("html PAGENAME\n")
-		fmt.Print("  Print the HTML of the page.\n")
-		fmt.Print("search TERMS\n")
-		fmt.Print("  Print the titles of the page with score.\n")
-	}
-}

concurrency_test.go

@@ -10,6 +10,6 @@ func TestLoadAndSearch(t *testing.T) {
 	index.reset()
 	go index.load()
 	q := "Oddµ"
-	pages := search(q)
+	pages, _, _ := search(q, 1)
 	assert.Zero(t, len(pages))
 }

edit.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
     <meta charset="utf-8">
     <meta name="format-detection" content="telephone=no">
@@ -15,7 +15,7 @@ form, textarea { width: 100%; }
     <form action="/save/{{.Name}}" method="POST">
       <textarea name="body" rows="20" cols="80" placeholder="# Title
 
-Text" autofocus>{{printf "%s" .Body}}</textarea>
+Text" lang="" autofocus>{{printf "%s" .Body}}</textarea>
       <p><input type="submit" value="Save">
         <a href="/view/{{.Name}}"><button type="button">Cancel</button></a></p>
     </form>

feed.go

@@ -0,0 +1,73 @@
+package main
+
+import (
+	"github.com/gomarkdown/markdown"
+	"github.com/gomarkdown/markdown/ast"
+	"github.com/gomarkdown/markdown/parser"
+	"html/template"
+	"bytes"
+	"os"
+	"path"
+	"time"
+)
+
+type Item struct {
+	Page
+	Date string
+}
+
+type Feed struct {
+	Item
+	Items []Item
+}
+
+func feed(p *Page, ti time.Time) *Feed {
+	feed := new(Feed)
+	feed.Name = p.Name
+	feed.Title = p.Title
+	feed.Date = ti.Format(time.RFC1123Z)
+	parser := parser.New()
+	doc := markdown.Parse(p.Body, parser)
+	items := make([]Item, 0)
+	inListItem := false
+	ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
+		// set the flag if we're in a list item
+		listItem, ok := node.(*ast.ListItem)
+		if ok && listItem.BulletChar == '*' {
+			inListItem = entering
+			return ast.GoToNext
+		}
+		// if we're not in a list item, continue
+		if !inListItem || !entering {
+			return ast.GoToNext
+		}
+		// if we're in a link and it's local
+		link, ok := node.(*ast.Link)
+		if !ok || bytes.Contains(link.Destination, []byte("//")) {
+			return ast.GoToNext
+		}
+		name := path.Join(path.Dir(p.Name), string(link.Destination))
+		fi, err := os.Stat(name + ".md")
+		if err != nil {
+			return ast.GoToNext
+		}
+		p2, err := loadPage(name)
+		if err != nil {
+			return ast.GoToNext
+		}
+		p2.handleTitle(false)
+		p2.renderHtml()
+		it := Item{Date: fi.ModTime().Format(time.RFC1123Z)}
+		it.Title = p2.Title
+		it.Name = p2.Name
+		it.Html = template.HTML(template.HTMLEscaper(p2.Html))
+		it.Hashtags = p2.Hashtags
+		items = append(items, it)
+		if len(items) >= 10 {
+			return ast.Terminate
+		}
+		return ast.GoToNext
+	})
+	feed.Items = items
+	return feed
+}

feed.html

@@ -0,0 +1,28 @@
+<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
+  <channel>
+    <docs>http://blogs.law.harvard.edu/tech/rss</docs>
+    <title>{{.Title}}</title>
+    <link>https://example.org/</link>
+    <managingEditor>you@example.org (Your Name)</managingEditor>
+    <webMaster>you@example.org (Your Name)</webMaster>
+    <atom:link href="https://example.org/view/{{.Name}}.rss" rel="self" type="application/rss+xml"/>
+    <description>This is the digital garden of Your Name.</description>
+    <image>
+      <url>https://example.org/view/logo.jpg</url>
+      <title>{{.Title}}</title>
+      <link>https://example.org/</link>
+    </image>
+    {{range .Items}}
+    <item>
+      <title>{{.Title}}</title>
+      <link>https://example.org/view/{{.Name}}</link>
+      <guid>https://example.org/view/{{.Name}}</guid>
+      <description>{{.Html}}</description>
+      <pubDate>{{.Date}}</pubDate>
+      {{range .Hashtags}}
+      <category>{{.}}</category>
+      {{end}}
+    </item>
+    {{end}}
+  </channel>
+</rss>

feed_test.go

@@ -0,0 +1,55 @@
+package main
+
+import (
+	"github.com/stretchr/testify/assert"
+	"os"
+	"testing"
+)
+
+func TestFeed(t *testing.T) {
+	assert.Contains(t, 
+		assert.HTTPBody(makeHandler(viewHandler, true), "GET", "/view/index.rss", nil),
+		"Welcome to Oddµ")
+}
+
+// wipes testdata
+func TestFeedItems(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	index.load()
+	
+	p1 := &Page{Name: "testdata/cactus", Body: []byte(`# Cactus
+Green head and white hair
+A bench in the evening sun
+Unmoved by the news
+
+#Succulent`)}
+	p1.save()
+
+	p2 := &Page{Name: "testdata/dragon", Body: []byte(`# Dragon
+My palm tree grows straight
+Up and up to touch the sky
+Ignoring the roof
+
+#Palmtree`)}
+	p2.save()
+
+	p3 := &Page{Name: "testdata/plants", Body: []byte(`# Plants
+Writing poems about plants.
+
+* [My Cactus](cactus)
+* [My Dragon Tree](dragon)`)}
+	p3.save()
+
+	body := assert.HTTPBody(makeHandler(viewHandler, true), "GET", "/view/testdata/plants.rss", nil)
+	assert.Contains(t, body, "<title>Plants</title>")
+	assert.Contains(t, body, "<title>Cactus</title>")
+	assert.Contains(t, body, "<title>Dragon</title>")
+	assert.Contains(t, body, "&lt;h1&gt;Cactus&lt;/h1&gt;")
+	assert.Contains(t, body, "&lt;h1&gt;Dragon&lt;/h1&gt;")
+	assert.Contains(t, body, "<category>Succulent</category>")
+	assert.Contains(t, body, "<category>Palmtree</category>")
+
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}

go.mod

@@ -3,8 +3,11 @@ module alexschroeder.ch/cgit/oddmu
 go 1.21.0
 
 require (
+	github.com/anthonynsimon/bild v0.13.0
 	github.com/dgryski/go-trigram v0.0.0-20160407183937-79ec494e1ad0
 	github.com/gomarkdown/markdown v0.0.0-20230912175223-14b07df9d538
+	github.com/google/subcommands v1.2.0
+	github.com/hexops/gotextdiff v1.0.3
 	github.com/microcosm-cc/bluemonday v1.0.25
 	github.com/pemistahl/lingua-go v1.4.0
 	github.com/stretchr/testify v1.8.4
@@ -14,10 +17,13 @@ require (
 	github.com/aymerick/douceur v0.2.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/gorilla/css v1.0.0 // indirect
+	github.com/kr/pretty v0.1.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/shopspring/decimal v1.3.1 // indirect
 	golang.org/x/exp v0.0.0-20221106115401-f9659909a136 // indirect
+	golang.org/x/image v0.0.0-20190703141733-d6a02ce849c9 // indirect
 	golang.org/x/net v0.12.0 // indirect
 	google.golang.org/protobuf v1.31.0 // indirect
+	gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )

go.sum

@@ -1,38 +1,77 @@
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/anthonynsimon/bild v0.13.0 h1:mN3tMaNds1wBWi1BrJq0ipDBhpkooYfu7ZFSMhXt1C8=
+github.com/anthonynsimon/bild v0.13.0/go.mod h1:tpzzp0aYkAsMi1zmfhimaDyX1xjn2OUc1AJZK/TF0AE=
+github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
 github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
 github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
+github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
+github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
+github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dgryski/go-trigram v0.0.0-20160407183937-79ec494e1ad0 h1:b+7JSiBM+hnLQjP/lXztks5hnLt1PS46hktG9VOJgzo=
 github.com/dgryski/go-trigram v0.0.0-20160407183937-79ec494e1ad0/go.mod h1:qzKC/DpcxK67zaSHdCmIv3L9WJViHVinYXN2S7l3RM8=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
 github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
-github.com/gomarkdown/markdown v0.0.0-20230716120725-531d2d74bc12 h1:uK3X/2mt4tbSGoHvbLBHUny7CKiuwUip3MArtukol4E=
-github.com/gomarkdown/markdown v0.0.0-20230716120725-531d2d74bc12/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA=
 github.com/gomarkdown/markdown v0.0.0-20230912175223-14b07df9d538 h1:ePDpFu7l0QUV46/9A7icfL2wvIOzTJLCWh4RO2NECzE=
 github.com/gomarkdown/markdown v0.0.0-20230912175223-14b07df9d538/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA=
 github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
 github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/subcommands v1.2.0 h1:vWQspBTo2nEqTUFita5/KeEWlUL8kQObDFbub/EN9oE=
+github.com/google/subcommands v1.2.0/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk=
 github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
 github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
+github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
+github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
+github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
+github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
 github.com/microcosm-cc/bluemonday v1.0.25 h1:4NEwSfiJ+Wva0VxN5B8OwMicaJvD8r9tlJWm9rtloEg=
 github.com/microcosm-cc/bluemonday v1.0.25/go.mod h1:ZIOjCQp1OrzBBPIJmfX4qDYFuhU02nx4bn030ixfHLE=
+github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
+github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
 github.com/pemistahl/lingua-go v1.4.0 h1:ifYhthrlW7iO4icdubwlduYnmwU37V1sbNrwhKBR4rM=
 github.com/pemistahl/lingua-go v1.4.0/go.mod h1:ECuM1Hp/3hvyh7k8aWSqNCPlTxLemFZsRjocUf3KgME=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
 github.com/shopspring/decimal v1.3.1 h1:2Usl1nmF/WZucqkFZhnfFYxxxu8LG21F6nPQBE5gKV8=
 github.com/shopspring/decimal v1.3.1/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
+github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
+github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
+github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
+github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
+github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
+golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/exp v0.0.0-20221106115401-f9659909a136 h1:Fq7F/w7MAa1KJ5bt2aJ62ihqp9HDcRuyILskkpIAurw=
 golang.org/x/exp v0.0.0-20221106115401-f9659909a136/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
+golang.org/x/image v0.0.0-20190703141733-d6a02ce849c9 h1:uc17S921SPw5F2gJo7slQ3aqvr2RwpL7eb3+DZncu3s=
+golang.org/x/image v0.0.0-20190703141733-d6a02ce849c9/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/net v0.12.0 h1:cfawfvKITfUsFCeJIHJrbSxpeu/E81khclypR0GVT50=
 golang.org/x/net v0.12.0/go.mod h1:zEVYFnQC7m/vmpQFELhcD1EWkZlX69l4oqgmer6hfKA=
+golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
 google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

html_cmd.go

@@ -0,0 +1,56 @@
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"github.com/google/subcommands"
+	"io"
+	"os"
+)
+
+type htmlCmd struct {
+	useTemplate bool
+}
+
+func (*htmlCmd) Name() string     { return "html" }
+func (*htmlCmd) Synopsis() string { return "Render a page as HTML." }
+func (*htmlCmd) Usage() string {
+	return `html [-view] <page name>:
+  Render a page as HTML.
+`
+}
+
+func (cmd *htmlCmd) SetFlags(f *flag.FlagSet) {
+	f.BoolVar(&cmd.useTemplate, "view", false, "Use the 'view.html' template.")
+}
+
+func (cmd *htmlCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+	return htmlCli(os.Stdout, cmd.useTemplate, f.Args())
+}
+
+func htmlCli(w io.Writer, useTemplate bool, args []string) subcommands.ExitStatus {
+		for _, arg := range args {
+		p, err := loadPage(arg)
+		if err != nil {
+			fmt.Fprintf(w, "Cannot load %s: %s\n", arg, err)
+			return subcommands.ExitFailure
+		}
+		initAccounts()
+		if useTemplate {
+			p.handleTitle(true)
+			p.renderHtml()
+			t := "view.html"
+			err := templates.ExecuteTemplate(w, t, p)
+			if err != nil {
+				fmt.Fprintf(w, "Cannot execute %s template for %s: %s\n", t, arg, err)
+				return subcommands.ExitFailure
+			}
+		} else {
+			// do not handle title
+			p.renderHtml()
+			fmt.Fprintln(w, p.Html)
+		}
+	}
+	return subcommands.ExitSuccess
+}

html_cmd_test.go

@@ -0,0 +1,22 @@
+package main
+
+import (
+	"bytes"
+	"github.com/stretchr/testify/assert"
+	"testing"
+	"github.com/google/subcommands"
+)
+
+func TestHtmlCmd(t *testing.T) {
+	b := new(bytes.Buffer)
+	s := htmlCli(b, false, []string{"index"})
+	assert.Equal(t, subcommands.ExitSuccess, s)
+	r := `<h1>Welcome to Oddµ</h1>
+
+<p>Hello! 🙃</p>
+
+<p>Check out the <a href="README" rel="nofollow">README</a>.</p>
+
+`
+	assert.Equal(t, r, b.String())
+}

index.go

@@ -21,6 +21,9 @@ type Index struct {
 	// documents is a map, mapping document ids of the index to
 	// page names.
 	documents map[trigram.DocID]string
+
+	// names is a map, mapping page names to titles.
+	titles map[string]string
 }
 
 // idx is the global Index per wiki.
@@ -30,6 +33,7 @@ var index Index
 func (idx *Index) reset() {
 	idx.index = nil
 	idx.documents = nil
+	idx.titles = nil
 }
 
 // add reads a file and adds it to the index. This must happen while
@@ -47,8 +51,10 @@ func (idx *Index) add(path string, info fs.FileInfo, err error) error {
 	if err != nil {
 		return err
 	}
+	p.handleTitle(false)
 	id := idx.index.Add(strings.ToLower(string(p.Body)))
 	idx.documents[id] = p.Name
+	idx.titles[p.Name] = p.Title
 	return nil
 }
 
@@ -59,6 +65,7 @@ func (idx *Index) load() (int, error) {
 	defer idx.Unlock()
 	idx.index = make(trigram.Index)
 	idx.documents = make(map[trigram.DocID]string)
+	idx.titles = make(map[string]string)
 	err := filepath.Walk(".", idx.add)
 	if err != nil {
 		idx.reset()
@@ -90,23 +97,27 @@ func (p *Page) updateIndex() {
 		o, err := loadPage(p.Name)
 		if err == nil {
 			index.index.Delete(strings.ToLower(string(o.Body)), id)
+			o.handleTitle(false)
+			delete(index.titles, o.Title)
 		}
 		index.index.Insert(strings.ToLower(string(p.Body)), id)
+		p.handleTitle(false)
+		index.titles[p.Name] = p.Title
 	}
 }
 
+// searchDocuments searches the index for a string. This requires the
+// index to be locked.
 func searchDocuments(q string) []string {
-		words := strings.Fields(strings.ToLower(q))
+	words := strings.Fields(strings.ToLower(q))
 	var trigrams []trigram.T
 	for _, word := range words {
 		trigrams = trigram.Extract(word, trigrams)
 	}
-	index.RLock()
 	ids := index.index.QueryTrigrams(trigrams)
 	names := make([]string, len(ids))
 	for i, id := range ids {
 		names[i] = index.documents[id]
 	}
-	index.RUnlock()
 	return names
 }

index_test.go

@@ -11,19 +11,19 @@ import (
 func TestIndex(t *testing.T) {
 	index.load()
 	q := "Oddµ"
-	pages := search(q)
+	pages, _, _ := search(q, 1)
 	assert.NotZero(t, len(pages))
 	for _, p := range pages {
 		assert.NotContains(t, p.Title, "<b>")
 		assert.True(t, strings.Contains(string(p.Body), q) || strings.Contains(string(p.Title), q))
-		assert.NotZero(t, p.Score)
+		assert.NotZero(t, p.Score, "Score %d for %s", p.Score, p.Name)
 	}
 }
 
 func TestSearchHashtag(t *testing.T) {
 	index.load()
-	q := "#Another_Tag"
-	pages := search(q)
+	q := "#like_this"
+	pages, _, _ := search(q, 1)
 	assert.NotZero(t, len(pages))
 }
 
@@ -35,7 +35,7 @@ func TestIndexUpdates(t *testing.T) {
 	p.save()
 
 	// Find the phrase
-	pages := search("This is a test")
+	pages, _, _ := search("This is a test", 1)
 	found := false
 	for _, p := range pages {
 		if p.Name == name {
@@ -46,7 +46,7 @@ func TestIndexUpdates(t *testing.T) {
 	assert.True(t, found)
 
 	// Find the phrase, case insensitive
-	pages = search("this is a test")
+	pages, _, _ = search("this is a test", 1)
 	found = false
 	for _, p := range pages {
 		if p.Name == name {
@@ -57,7 +57,7 @@ func TestIndexUpdates(t *testing.T) {
 	assert.True(t, found)
 
 	// Find some words
-	pages = search("this test")
+	pages, _, _ = search("this test", 1)
 	found = false
 	for _, p := range pages {
 		if p.Name == name {
@@ -70,7 +70,7 @@ func TestIndexUpdates(t *testing.T) {
 	// Update the page and no longer find it with the old phrase
 	p = &Page{Name: name, Body: []byte("Guvf vf n grfg.")}
 	p.save()
-	pages = search("This is a test")
+	pages, _, _ = search("This is a test", 1)
 	found = false
 	for _, p := range pages {
 		if p.Name == name {
@@ -81,7 +81,7 @@ func TestIndexUpdates(t *testing.T) {
 	assert.False(t, found)
 
 	// Find page using a new word
-	pages = search("Guvf")
+	pages, _, _ = search("Guvf", 1)
 	found = false
 	for _, p := range pages {
 		if p.Name == name {

man/Makefile

@@ -0,0 +1,5 @@
+docs: oddmu.1 oddmu.5 oddmu-templates.5 oddmu-apache.5 oddmu.service.5 oddmu-replace.1 \
+	oddmu-search.1 oddmu-search.7 oddmu-html.1
+
+oddmu%: oddmu%.txt
+	scdoc < $< > $@

man/oddmu-apache.5

@@ -0,0 +1,220 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-APACHE" "5" "2023-09-18"
+.PP
+.SH NAME
+.PP
+oddmu-apache - how to setup Apache as a reverse proxy for Oddmu
+.PP
+.SS DESCRIPTION
+.PP
+The oddmu program serves the current working directory as a wiki on
+port 8080.\& This is an unpriviledged port so an ordinary use account
+can do this.\&
+.PP
+The best way to protect the wiki against vandalism and spam is to use
+a regular web server as reverse proxy.\& This page explains how to setup
+Apache on Debian to do this.\&
+.PP
+.SS CONFIGURATION
+.PP
+HTTPS is not part of the wiki.\& You probably want to configure this in
+your webserver.\& I guess you could use stunnel, too.\& If you'\&re using
+Apache, you can use "mod_md" to manage your domain.\&
+.PP
+In the example below, the site is configured in a file called
+"/etc/apache2/sites-available/500-transjovian.\&conf" and a link poins
+there from "/etc/apache2/sites-enabled".\& Create this link using
+\fIa2ensite\fR(1).\&
+.PP
+.nf
+.RS 4
+MDomain transjovian\&.org
+MDCertificateAgreement accepted
+
+<VirtualHost *:80>
+    ServerName transjovian\&.org
+    RewriteEngine on
+    RewriteRule ^/(\&.*) https://%{HTTP_HOST}/$1 [redirect]
+</VirtualHost>
+<VirtualHost *:443>
+    ServerAdmin alex@alexschroeder\&.ch
+    ServerName transjovian\&.org
+    SSLEngine on
+    ProxyPassMatch ^/(search|(view|edit|save|add|append|upload|drop)/(\&.*))?$ http://localhost:8080/$1
+</VirtualHost>
+.fi
+.RE
+.PP
+First, it manages the domain, getting the necessary certificates.\& It
+redirects regular HTTP traffic from port 80 to port 443.\& It turns on
+the SSL engine for port 443.\& It proxies the requests for the wiki to
+port 8080.\&
+.PP
+Thus, this is what happens:
+.PP
+\fB The user tells the browser to visit `transjovian.\&org`
+\fR The browser sends a request for `http://transjovian.\&org` (on port 80)
+\fB Apache redirects this to `https://transjovian.\&org/` by default (now on port 443)
+\fR This is proxied to `http://transjovian.\&org:8080/` (no encryption, on port 8080)
+.PP
+Restart the server, gracefully:
+.PP
+.nf
+.RS 4
+apachectl graceful
+.fi
+.RE
+.PP
+To serve both HTTP and HTTPS, don'\&t redirect from the first virtual
+host to the second – instead just proxy to the wiki like you did for
+the second virtual host: use a copy of the "ProxyPassMatch" directive
+instead of "RewriteEngine on" and "RewriteRule".\&
+.PP
+.SS Access
+.PP
+Access control is not part of the wiki.\& By default, the wiki is
+editable by all.\& This is most likely not what you want unless you'\&re
+running it stand-alone, unconnected to the Internet.\&
+.PP
+Create a new password file called ".\&htpasswd" and add the user "alex":
+.PP
+.nf
+.RS 4
+cd /home/oddmu
+htpasswd -c \&.htpasswd alex
+.fi
+.RE
+.PP
+To add more users, don'\&t use the "-c" option or you will overwrite it!\&
+.PP
+To add another user:
+.PP
+.nf
+.RS 4
+htpasswd \&.htpasswd berta
+.fi
+.RE
+.PP
+To remove a user:
+.PP
+.nf
+.RS 4
+htpasswd -D \&.htpasswd berta
+.fi
+.RE
+.PP
+Modify your site configuration and protect the "/edit/", "/save/",
+"/add/", "/append/", "/upload/" and "/drop/" URLs with a password by
+adding the following to your "<VirtualHost *:443>" section:
+.PP
+.nf
+.RS 4
+<LocationMatch "^/(edit|save|add|append|upload|drop)/">
+  AuthType Basic
+  AuthName "Password Required"
+  AuthUserFile /home/oddmu/\&.htpasswd
+  Require valid-user
+</LocationMatch>
+.fi
+.RE
+.PP
+.SS Serve static files
+.PP
+If you want to serve static files as well, add a document root to your
+webserver configuration.\& In this case, the document root is the
+directory where all the data files are.\& Apache will not serve files
+such as ".\&htpasswd".\&
+.PP
+.nf
+.RS 4
+DocumentRoot /home/oddmu
+<Directory /home/oddmu>
+    Require all granted
+</Directory>
+.fi
+.RE
+.PP
+Make sure that none of the subdirectories look like the wiki paths
+"/view/", "/edit/", "/save/", "/add/", "/append/", "/upload/",
+"/drop/" or "/search".\& For example, create a file called "robots.\&txt"
+containing the following, telling all robots that they'\&re not welcome.\&
+.PP
+.nf
+.RS 4
+User-agent: *
+Disallow: /
+.fi
+.RE
+.PP
+You site now serves "/robots.\&txt" without interfering with the wiki,
+and without needing a wiki page.\&
+.PP
+.SS Different logins for different access rights
+.PP
+What if you have a site with various subdirectories and each
+subdirectory is for a different group of friends?\& You can set this up
+using your webserver.\& One way to do this is to require specific
+usernames (which must have a password in the password file mentioned
+above.\&
+.PP
+This requires a valid login by the user "alex" or "berta":
+.PP
+.nf
+.RS 4
+<LocationMatch "^/(edit|save|add|append|upload|drop)/intetebi/">
+  Require user alex berta
+</LocationMatch>
+.fi
+.RE
+.PP
+.SS Private wikis
+.PP
+Based on the above, you can prevent people from \fIreading\fR the wiki.\&
+The "LocationMatch" must cover the "/view/" URLs in order to protect
+\fBeverything\fR.\&
+.PP
+.nf
+.RS 4
+<Location />
+  AuthType Basic
+  AuthName "Password Required"
+  AuthUserFile /home/oddmu/\&.htpasswd
+  Require valid-user
+</Location>
+.fi
+.RE
+.PP
+.SS Virtual hosting
+.PP
+Virtual hosting in this context means that the program serves two
+different sites for two different domains from the same machine.\& Oddmu
+doesn'\&t support that, but your webserver does.\& Therefore, start an
+Oddmu instance for every domain name, each listening on a different
+port.\& Then set up your web server such that ever domain acts as a
+reverse proxy to a different Oddmu instance.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1)
+.PP
+"Apache Core Features".\&
+https://httpd.\&apache.\&org/docs/current/mod/core.\&html
+.PP
+"Apache: Authentication and Authorization".\&
+https://httpd.\&apache.\&org/docs/current/howto/auth.\&html
+.PP
+"Apache Module mod_proxy".\&
+https://httpd.\&apache.\&org/docs/current/mod/mod_proxy.\&html
+.PP
+"Robot exclusion standard" on Wikipedia.\&
+https://en.\&wikipedia.\&org/wiki/Robot_exclusion_standard
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-apache.5.txt

@@ -0,0 +1,193 @@
+ODDMU-APACHE(5)
+
+# NAME
+
+oddmu-apache - how to setup Apache as a reverse proxy for Oddmu
+
+## DESCRIPTION
+
+The oddmu program serves the current working directory as a wiki on
+port 8080. This is an unpriviledged port so an ordinary use account
+can do this.
+
+The best way to protect the wiki against vandalism and spam is to use
+a regular web server as reverse proxy. This page explains how to setup
+Apache on Debian to do this.
+
+## CONFIGURATION
+
+HTTPS is not part of the wiki. You probably want to configure this in
+your webserver. I guess you could use stunnel, too. If you're using
+Apache, you can use "mod_md" to manage your domain.
+
+In the example below, the site is configured in a file called
+"/etc/apache2/sites-available/500-transjovian.conf" and a link poins
+there from "/etc/apache2/sites-enabled". Create this link using
+_a2ensite_(1).
+
+```
+MDomain transjovian.org
+MDCertificateAgreement accepted
+
+<VirtualHost *:80>
+    ServerName transjovian.org
+    RewriteEngine on
+    RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [redirect]
+</VirtualHost>
+<VirtualHost *:443>
+    ServerAdmin alex@alexschroeder.ch
+    ServerName transjovian.org
+    SSLEngine on
+    ProxyPassMatch ^/(search|(view|edit|save|add|append|upload|drop)/(.*))?$ http://localhost:8080/$1
+</VirtualHost>
+```
+
+First, it manages the domain, getting the necessary certificates. It
+redirects regular HTTP traffic from port 80 to port 443. It turns on
+the SSL engine for port 443. It proxies the requests for the wiki to
+port 8080.
+
+Thus, this is what happens:
+
+* The user tells the browser to visit `transjovian.org`
+* The browser sends a request for `http://transjovian.org` (on port 80)
+* Apache redirects this to `https://transjovian.org/` by default (now on port 443)
+* This is proxied to `http://transjovian.org:8080/` (no encryption, on port 8080)
+
+Restart the server, gracefully:
+
+```
+apachectl graceful
+```
+
+To serve both HTTP and HTTPS, don't redirect from the first virtual
+host to the second – instead just proxy to the wiki like you did for
+the second virtual host: use a copy of the "ProxyPassMatch" directive
+instead of "RewriteEngine on" and "RewriteRule".
+
+## Access
+
+Access control is not part of the wiki. By default, the wiki is
+editable by all. This is most likely not what you want unless you're
+running it stand-alone, unconnected to the Internet.
+
+Create a new password file called ".htpasswd" and add the user "alex":
+
+```
+cd /home/oddmu
+htpasswd -c .htpasswd alex
+```
+
+To add more users, don't use the "-c" option or you will overwrite it!
+
+To add another user:
+
+```
+htpasswd .htpasswd berta
+```
+
+To remove a user:
+
+```
+htpasswd -D .htpasswd berta
+```
+
+Modify your site configuration and protect the "/edit/", "/save/",
+"/add/", "/append/", "/upload/" and "/drop/" URLs with a password by
+adding the following to your "<VirtualHost \*:443>" section:
+
+```
+<LocationMatch "^/(edit|save|add|append|upload|drop)/">
+  AuthType Basic
+  AuthName "Password Required"
+  AuthUserFile /home/oddmu/.htpasswd
+  Require valid-user
+</LocationMatch>
+```
+
+## Serve static files
+
+If you want to serve static files as well, add a document root to your
+webserver configuration. In this case, the document root is the
+directory where all the data files are. Apache will not serve files
+such as ".htpasswd".
+
+```
+DocumentRoot /home/oddmu
+<Directory /home/oddmu>
+    Require all granted
+</Directory>
+```
+
+Make sure that none of the subdirectories look like the wiki paths
+"/view/", "/edit/", "/save/", "/add/", "/append/", "/upload/",
+"/drop/" or "/search". For example, create a file called "robots.txt"
+containing the following, telling all robots that they're not welcome.
+
+```
+User-agent: *
+Disallow: /
+```
+
+You site now serves "/robots.txt" without interfering with the wiki,
+and without needing a wiki page.
+
+## Different logins for different access rights
+
+What if you have a site with various subdirectories and each
+subdirectory is for a different group of friends? You can set this up
+using your webserver. One way to do this is to require specific
+usernames (which must have a password in the password file mentioned
+above.
+
+This requires a valid login by the user "alex" or "berta":
+
+```
+<LocationMatch "^/(edit|save|add|append|upload|drop)/intetebi/">
+  Require user alex berta
+</LocationMatch>
+```
+
+## Private wikis
+
+Based on the above, you can prevent people from _reading_ the wiki.
+The "LocationMatch" must cover the "/view/" URLs in order to protect
+*everything*.
+
+```
+<Location />
+  AuthType Basic
+  AuthName "Password Required"
+  AuthUserFile /home/oddmu/.htpasswd
+  Require valid-user
+</Location>
+```
+
+## Virtual hosting
+
+Virtual hosting in this context means that the program serves two
+different sites for two different domains from the same machine. Oddmu
+doesn't support that, but your webserver does. Therefore, start an
+Oddmu instance for every domain name, each listening on a different
+port. Then set up your web server such that ever domain acts as a
+reverse proxy to a different Oddmu instance.
+
+# SEE ALSO
+
+_oddmu_(1)
+
+"Apache Core Features".
+https://httpd.apache.org/docs/current/mod/core.html
+
+"Apache: Authentication and Authorization".
+https://httpd.apache.org/docs/current/howto/auth.html
+
+"Apache Module mod_proxy".
+https://httpd.apache.org/docs/current/mod/mod_proxy.html
+
+"Robot exclusion standard" on Wikipedia.
+https://en.wikipedia.org/wiki/Robot_exclusion_standard
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-html.1

@@ -0,0 +1,53 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-HTML" "1" "2023-09-22"
+.PP
+.SH NAME
+.PP
+oddmu-html - render Oddmu page HTML from the command-line
+.PP
+.SH SYNOPSIS
+.PP
+\fBoddmu html\fR [-view] \fIpage-name\fR
+.PP
+.SH DESCRIPTION
+.PP
+The "html" subcommand opens the Markdown file for the given page name
+(appending the ".\&md" extension) and prints the HTML to STDOUT without
+invoking the "view.\&html" template.\&
+.PP
+.SH OPTIONS
+.PP
+\fB-view\fR
+.RS 4
+Use the "view.\&html" template to render the page.\& Without this,
+the HTML will lack html and body tags.\&
+.PP
+.RE
+.SH EXAMPLE
+.PP
+Generate the HTML for "README.\&md":
+.PP
+.nf
+.RS 4
+oddmu html README
+.fi
+.RE
+.PP
+.SH ENVIRONMENT
+.PP
+The ODDMU_WEBFINGER environment variable has no effect in this
+situation.\& Fediverse accounts are not linked to their profile pages.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-html.1.txt

@@ -0,0 +1,42 @@
+ODDMU-HTML(1)
+
+# NAME
+
+oddmu-html - render Oddmu page HTML from the command-line
+
+# SYNOPSIS
+
+*oddmu html* [-view] _page-name_
+
+# DESCRIPTION
+
+The "html" subcommand opens the Markdown file for the given page name
+(appending the ".md" extension) and prints the HTML to STDOUT without
+invoking the "view.html" template.
+
+# OPTIONS
+
+*-view*
+	Use the "view.html" template to render the page. Without this,
+	the HTML will lack html and body tags.
+
+# EXAMPLE
+
+Generate the HTML for "README.md":
+
+```
+oddmu html README
+```
+
+# ENVIRONMENT
+
+The ODDMU_WEBFINGER environment variable has no effect in this
+situation. Fediverse accounts are not linked to their profile pages.
+
+# SEE ALSO
+
+_oddmu_(1)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-replace.1

@@ -0,0 +1,66 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-REPLACE" "1" "2023-09-22"
+.PP
+.SH NAME
+.PP
+oddmu-replace - replace text in Oddmu pages from the command-line
+.PP
+.SH SYNOPSIS
+.PP
+\fBoddmu replace\fR [-confirm] \fIregexp\fR \fIreplacement\fR
+.PP
+.SH DESCRIPTION
+.PP
+The "replace" subcommand replaces the Markdown files in the current
+directory (!\&), returning the replace result as a Markdown-formatted
+list.\&
+.PP
+.SH OPTIONS
+.PP
+\fB-confirm\fR
+.RS 4
+By default, the replacement doesn'\&t save the changes made.\&
+Instead, a unified diff is produced and printed.\& Given this
+option, the changed Markdown files are saved to disk.\&
+.PP
+.RE
+.SH EXAMPLE
+.PP
+Replace for "oddmu" in the Markdown files of the current directory:
+.PP
+.nf
+.RS 4
+oddmu replace oddmu
+.fi
+.RE
+.PP
+Result:
+.PP
+.nf
+.RS 4
+Replace oddmu: 1 result
+* [Oddµ: A minimal wiki](README) (5)
+.fi
+.RE
+.PP
+.SH NOTES
+.PP
+This is the equivalent of using \fIsed\fR(1) with the --quiet,
+--regexp-extended, --in-place=~ and --expression command with the s
+command "s/regexp/replacement/g" except that it prints a unified diff
+per default instead of making any changes and the regexp rules differ
+slightly.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1), \fIoddmu-search\fR(7)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-replace.1.txt

@@ -0,0 +1,53 @@
+ODDMU-REPLACE(1)
+
+# NAME
+
+oddmu-replace - replace text in Oddmu pages from the command-line
+
+# SYNOPSIS
+
+*oddmu replace* [-confirm] _regexp_ _replacement_
+
+# DESCRIPTION
+
+The "replace" subcommand replaces the Markdown files in the current
+directory (!), returning the replace result as a Markdown-formatted
+list.
+
+# OPTIONS
+
+*-confirm*
+	By default, the replacement doesn't save the changes made.
+	Instead, a unified diff is produced and printed. Given this
+	option, the changed Markdown files are saved to disk.
+
+# EXAMPLE
+
+Replace for "oddmu" in the Markdown files of the current directory:
+
+```
+oddmu replace oddmu
+```
+
+Result:
+
+```
+Replace oddmu: 1 result
+* [Oddµ: A minimal wiki](README) (5)
+```
+
+# NOTES
+
+This is the equivalent of using _sed_(1) with the --quiet,
+\--regexp-extended, --in-place=~ and --expression command with the s
+command "s/regexp/replacement/g" except that it prints a unified diff
+per default instead of making any changes and the regexp rules differ
+slightly.
+
+# SEE ALSO
+
+_oddmu_(1), _oddmu-search_(7)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-search.1

@@ -0,0 +1,61 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-SEARCH" "1" "2023-09-19"
+.PP
+.SH NAME
+.PP
+oddmu-search - search the Oddmu pages from the command-line
+.PP
+.SH SYNOPSIS
+.PP
+\fBoddmu search\fR [-page \fIn\fR] \fIterms.\&.\&.\&\fR
+.PP
+.SH DESCRIPTION
+.PP
+The "search" subcommand searches the Markdown files in the current
+directory (!\&), returning the search result as a Markdown-formatted
+list.\&
+.PP
+The use of a trigram index makes it possible to find substrings and
+for the word order not to matter, but it also makes the search results
+a bit harder to understand.\& See \fIoddmu-search\fR(7) for more.\&
+.PP
+.SH OPTIONS
+.PP
+\fB-page\fR \fIn\fR
+.RS 4
+Search results are paginated and by default only the first
+page is shown.\& This option allows you to view other pages.\&
+.PP
+.RE
+.SH EXAMPLE
+.PP
+Search for "oddmu" in the Markdown files of the current directory:
+.PP
+.nf
+.RS 4
+oddmu search oddmu
+.fi
+.RE
+.PP
+Result:
+.PP
+.nf
+.RS 4
+Search oddmu: 1 result
+* [Oddµ: A minimal wiki](README) (5)
+.fi
+.RE
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1), \fIoddmu-replace\fR(1), \fIoddmu-search\fR(7)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-search.1.txt

@@ -0,0 +1,48 @@
+ODDMU-SEARCH(1)
+
+# NAME
+
+oddmu-search - search the Oddmu pages from the command-line
+
+# SYNOPSIS
+
+*oddmu search* [-page _n_] _terms..._
+
+# DESCRIPTION
+
+The "search" subcommand searches the Markdown files in the current
+directory (!), returning the search result as a Markdown-formatted
+list.
+
+The use of a trigram index makes it possible to find substrings and
+for the word order not to matter, but it also makes the search results
+a bit harder to understand. See _oddmu-search_(7) for more.
+
+# OPTIONS
+
+*-page* _n_
+	Search results are paginated and by default only the first
+	page is shown. This option allows you to view other pages.
+
+# EXAMPLE
+
+Search for "oddmu" in the Markdown files of the current directory:
+
+```
+oddmu search oddmu
+```
+
+Result:
+
+```
+Search oddmu: 1 result
+* [Oddµ: A minimal wiki](README) (5)
+```
+
+# SEE ALSO
+
+_oddmu_(1), _oddmu-replace_(1), _oddmu-search_(7)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-search.7

@@ -0,0 +1,89 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-SEARCH" "7" "2023-09-18"
+.PP
+.SH NAME
+.PP
+oddmu-search - understanding the Oddmu search engine
+.PP
+.SH SYNOPSIS
+.PP
+\fBoddmu search\fR \fIterms\fR.\&.\&.\&
+.PP
+.SH DESCRIPTION
+.PP
+The index indexes trigrams.\& Each group of three characters is a
+trigram.\& A document with content "This is a test" is turned to lower
+case and indexed under the trigrams "thi", "his", "is ", "s i", " is",
+"is ", "s a", " a ", "a t", " te", "tes", "est".\&
+.PP
+Each query is split into words and then processed the same way.\& A
+query with the words "this test" is turned to lower case and produces
+the trigrams "thi", "his", "tes", "est".\& This means that the word
+order is not considered when searching for documents.\&
+.PP
+This also means that there is no stemming.\& Searching for "testing"
+won'\&t find "This is a test" because there are no matches for the
+trigrams "sti", "tin", "ing".\&
+.PP
+These trigrams are looked up in the index, resulting in the list of
+documents.\& Each document found is then scored.\& Each of the following
+increases the score by one point:
+.PP
+.PD 0
+.IP \(bu 4
+the entire phrase matches
+.IP \(bu 4
+a word matches
+.IP \(bu 4
+a word matches at the beginning of a word
+.IP \(bu 4
+a word matches at the end of a word
+.IP \(bu 4
+a word matches as a whole word
+.PD
+.PP
+A document with content "This is a test" when searched with the phrase
+"this test" therefore gets a score of 8: the entire phrase does not
+match but each word gets four points.\&
+.PP
+Trigrams are sometimes strange: In a text containing the words "main"
+and "rail", a search for "mail" returns a match because the trigrams
+"mai" and "ail" are found.\& In this situation, the result has a score
+of 0.\&
+.PP
+The sorting of all the pages, however, does not depend on scoring!\&
+Computing the score is expensive because the page must be loaded from
+disk.\& Therefore, results are sorted by title:
+.PP
+.PD 0
+.IP \(bu 4
+If the page title contains the query string, it gets sorted first.\&
+.IP \(bu 4
+If the page name (the filename!\&) begins with a number, it is sorted
+descending.\&
+.IP \(bu 4
+All other pages follow, sorted ascending.\&
+.PD
+.PP
+The effect is that first, the pages with matches in the page title are
+shown, and then all the others.\& Within these two groups, the most
+recent blog posts are shown first.\& This assumes that blog pages start
+with an ISO date like "2023-09-16".\&
+.PP
+The score and highlighting of snippets is used to help visitors decide
+which links to click.\& A score of 0 indicates that all the trigrams
+were found but \fIno exact matches\fR for any of the terms.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1), \fIoddmu-search\fR(1)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-search.7.txt

@@ -0,0 +1,70 @@
+ODDMU-SEARCH(7)
+
+# NAME
+
+oddmu-search - understanding the Oddmu search engine
+
+# SYNOPSIS
+
+*oddmu search* _terms_...
+
+# DESCRIPTION
+
+The index indexes trigrams. Each group of three characters is a
+trigram. A document with content "This is a test" is turned to lower
+case and indexed under the trigrams "thi", "his", "is ", "s i", " is",
+"is ", "s a", " a ", "a t", " te", "tes", "est".
+
+Each query is split into words and then processed the same way. A
+query with the words "this test" is turned to lower case and produces
+the trigrams "thi", "his", "tes", "est". This means that the word
+order is not considered when searching for documents.
+
+This also means that there is no stemming. Searching for "testing"
+won't find "This is a test" because there are no matches for the
+trigrams "sti", "tin", "ing".
+
+These trigrams are looked up in the index, resulting in the list of
+documents. Each document found is then scored. Each of the following
+increases the score by one point:
+
+- the entire phrase matches
+- a word matches
+- a word matches at the beginning of a word
+- a word matches at the end of a word
+- a word matches as a whole word
+
+A document with content "This is a test" when searched with the phrase
+"this test" therefore gets a score of 8: the entire phrase does not
+match but each word gets four points.
+
+Trigrams are sometimes strange: In a text containing the words "main"
+and "rail", a search for "mail" returns a match because the trigrams
+"mai" and "ail" are found. In this situation, the result has a score
+of 0.
+
+The sorting of all the pages, however, does not depend on scoring!
+Computing the score is expensive because the page must be loaded from
+disk. Therefore, results are sorted by title:
+
+- If the page title contains the query string, it gets sorted first.
+- If the page name (the filename!) begins with a number, it is sorted
+  descending.
+- All other pages follow, sorted ascending.
+
+The effect is that first, the pages with matches in the page title are
+shown, and then all the others. Within these two groups, the most
+recent blog posts are shown first. This assumes that blog pages start
+with an ISO date like "2023-09-16".
+
+The score and highlighting of snippets is used to help visitors decide
+which links to click. A score of 0 indicates that all the trigrams
+were found but _no exact matches_ for any of the terms.
+
+# SEE ALSO
+
+_oddmu_(1), _oddmu-search_(1)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-templates.5

@@ -0,0 +1,152 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-TEMPLATES" "5" "2023-09-22" "File Formats Manual"
+.PP
+.SH NAME
+.PP
+oddmu-templates - how to write the templates
+.PP
+.SH SYNTAX
+.PP
+The templates can refer to the following properties of a page:
+.PP
+\fI{{.\&Title}}\fR is the page title.\& If the page doesn'\&t provide its own
+title, the page name is used.\&
+.PP
+\fI{{.\&Name}}\fR is the page name, escaped for use in URLs.\& More
+specifically, it is URI escaped except for the slashes.\& The page name
+doesn'\&t include the \fI.\&md\fR extension.\&
+.PP
+For the \fIview.\&html\fR template:
+.PP
+\fI{{.\&Html}}\fR is the rendered Markdown, as HTML.\&
+.PP
+\fI{{.\&Hashtags}}\fR is an array of strings.\&
+.PP
+For the \fIedit.\&html\fR template:
+.PP
+\fI{{printf "%s" .\&Body}}\fR is the Markdown, as a string (the data itself
+is a byte array and that'\&s why we need to call \fIprintf\fR).\&
+.PP
+For the \fIsearch.\&html\fR template only:
+.PP
+\fI{{.\&Previous}}\fR, \fI{{.\&Page}}\fR, \fI{{.\&Next}}\fR and \fI{{.\&Last}}\fR are the
+previous, current, next and last page number in the results since
+doing arithmetics in templates is hard.\& The first page number is 1.\&
+.PP
+\fI{{.\&More}}\fR indicates if there are any more search results.\&
+.PP
+\fI{{.\&Results}}\fR indicates if there were any search results at all.\&
+.PP
+\fI{{.\&Items}}\fR is an array of pages, each containing a search result.\& A
+search result is a page (with the properties seen above).\& Thus, to
+refer to them, you need to use a \fI{{range .\&Items}}\fR … \fI{{end}}\fR
+construct.\&
+.PP
+For items in the search result:
+.PP
+\fI{{.\&Html}}\fR is the rendered Markdown of a page summary, as HTML.\&
+.PP
+\fI{{.\&Score}}\fR is a numerical score for search results.\&
+.PP
+For the \fIfeed.\&html\fR template:
+.PP
+\fI{{.\&Name}}\fR is the page name, escaped for use in URLs.\&
+.PP
+\fI{{.\&Title}}\fR is the title of the underlying main page.\&
+.PP
+\fI{{.\&Date}}\fR is the date of the last update to the underlying main page,
+in RFC 822 format.\&
+.PP
+\fI{{.\&Items}}\fR is an array of feed items.\&
+.PP
+For items in the feed:
+.PP
+\fI{{.\&Name}}\fR is the page name, escaped for use in URLs.\&
+.PP
+\fI{{.\&Title}}\fR is the title of the page.\&
+.PP
+\fI{{.\&Html}}\fR is the rendered Markdown, as HTML.\&
+.PP
+\fI{{.\&Hashtags}}\fR is an array of strings.\&
+.PP
+\fI{{.\&Date}}\fR, the date of the last update to this page, in RFC 822
+format.\&
+.PP
+The \fIupload.\&html\fR template cannot refer to anything.\&
+.PP
+When calling the \fIsave\fR and \fIappend\fR action, the page name is taken
+from the URL path and the page content is taken from the \fIbody\fR form
+parameter.\& To illustrate, here'\&s how to edit the "welcome" page using
+\fIcurl\fR:
+.PP
+.nf
+.RS 4
+curl --form body="Did you bring a towel?" 
+  http://localhost:8080/save/welcome
+.fi
+.RE
+.PP
+When calling the \fIsearch\fR action, the query is taken from the URL
+parameter \fIq\fR.\&
+.PP
+.nf
+.RS 4
+curl http://localhost:8080/search?q=towel
+.fi
+.RE
+.PP
+.SS Non-English hyphenation
+.PP
+Automatic hyphenation by the browser requires two things: The style
+sheet must indicate "hyphen: auto" for an HTML element such as "body",
+and that element must have a "lang" set (usually a two letter language
+code such as "de" for German).\&
+.PP
+Oddmu attempts to detect the correct language for each page.\& It
+assumes that languages are not mixed on the same page.\& If you know
+that you'\&re only going to use a small number of languages – or just a
+single language!\& – you can set the environment variable
+ODDMU_LANGUAGES to a comma-separated list of ISO 639-1 codes, e.\&g.\&
+"en" or "en,de,fr,pt".\&
+.PP
+"view.\&html" is used the template to render a single page and so the
+language detected is added to the "html" element.\&
+.PP
+"search.\&html" is the template used to render search results and so
+"en" is used for the "html" element and the language detected for
+every page in the search result is added to the "article" element for
+each snippet.\&
+.PP
+"edit.\&html" and "add.\&html" are the templates used to edit a page and
+at that point, the language isn'\&t known, so "en" is used for the
+"html" element and no language is used for the "textarea" element.\&
+.PP
+SEE ALSO
+.PP
+\fIoddmu\fR(1)
+.PP
+"Structuring the web with HTML"
+https://developer.\&mozilla.\&org/en-US/docs/Learn/HTML
+.PP
+"Learn to style HTML using CSS"
+https://developer.\&mozilla.\&org/en-US/docs/Learn/CSS
+.PP
+The "text/template" library explains how to write templates from a
+programmer perspective.\& https://pkg.\&go.\&dev/text/template
+.PP
+The "html/template" library explains how the templates are made more
+secure in a HTML context.\& https://pkg.\&go.\&dev/html/template
+.PP
+"Lingua" is the library used to detect languages.\&
+https://github.\&com/pemistahl/lingua-go
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\& Up-to-date sources can be
+found at https://alexschroeder.\&ch/cgit/oddmu/.\&

man/oddmu-templates.5.txt

@@ -0,0 +1,141 @@
+ODDMU-TEMPLATES(5) "File Formats Manual"
+
+# NAME
+
+oddmu-templates - how to write the templates
+
+# SYNTAX
+
+The templates can refer to the following properties of a page:
+
+_{{.Title}}_ is the page title. If the page doesn't provide its own
+title, the page name is used.
+
+_{{.Name}}_ is the page name, escaped for use in URLs. More
+specifically, it is URI escaped except for the slashes. The page name
+doesn't include the _.md_ extension.
+
+For the _view.html_ template:
+
+_{{.Html}}_ is the rendered Markdown, as HTML.
+
+_{{.Hashtags}}_ is an array of strings.
+
+For the _edit.html_ template:
+
+_{{printf "%s" .Body}}_ is the Markdown, as a string (the data itself
+is a byte array and that's why we need to call _printf_).
+
+For the _search.html_ template only:
+
+_{{.Previous}}_, _{{.Page}}_, _{{.Next}}_ and _{{.Last}}_ are the
+previous, current, next and last page number in the results since
+doing arithmetics in templates is hard. The first page number is 1.
+
+_{{.More}}_ indicates if there are any more search results.
+
+_{{.Results}}_ indicates if there were any search results at all.
+
+_{{.Items}}_ is an array of pages, each containing a search result. A
+search result is a page (with the properties seen above). Thus, to
+refer to them, you need to use a _{{range .Items}}_ … _{{end}}_
+construct.
+
+For items in the search result:
+
+_{{.Html}}_ is the rendered Markdown of a page summary, as HTML.
+
+_{{.Score}}_ is a numerical score for search results.
+
+For the _feed.html_ template:
+
+_{{.Name}}_ is the page name, escaped for use in URLs.
+
+_{{.Title}}_ is the title of the underlying main page.
+
+_{{.Date}}_ is the date of the last update to the underlying main page,
+in RFC 822 format.
+
+_{{.Items}}_ is an array of feed items.
+
+For items in the feed:
+
+_{{.Name}}_ is the page name, escaped for use in URLs.
+
+_{{.Title}}_ is the title of the page.
+
+_{{.Html}}_ is the rendered Markdown, as escaped (!) HTML.
+
+_{{.Hashtags}}_ is an array of strings.
+
+_{{.Date}}_, the date of the last update to this page, in RFC 822
+format.
+
+The _upload.html_ template cannot refer to anything.
+
+When calling the _save_ and _append_ action, the page name is taken
+from the URL path and the page content is taken from the _body_ form
+parameter. To illustrate, here's how to edit the "welcome" page using
+_curl_:
+
+```
+curl --form body="Did you bring a towel?" \
+  http://localhost:8080/save/welcome
+```
+
+When calling the _search_ action, the query is taken from the URL
+parameter _q_.
+
+```
+curl http://localhost:8080/search?q=towel
+```
+
+## Non-English hyphenation
+
+Automatic hyphenation by the browser requires two things: The style
+sheet must indicate "hyphen: auto" for an HTML element such as "body",
+and that element must have a "lang" set (usually a two letter language
+code such as "de" for German).
+
+Oddmu attempts to detect the correct language for each page. It
+assumes that languages are not mixed on the same page. If you know
+that you're only going to use a small number of languages – or just a
+single language! – you can set the environment variable
+ODDMU_LANGUAGES to a comma-separated list of ISO 639-1 codes, e.g.
+"en" or "en,de,fr,pt".
+
+"view.html" is used the template to render a single page and so the
+language detected is added to the "html" element.
+
+"search.html" is the template used to render search results and so
+"en" is used for the "html" element and the language detected for
+every page in the search result is added to the "article" element for
+each snippet.
+
+"edit.html" and "add.html" are the templates used to edit a page and
+at that point, the language isn't known, so "en" is used for the
+"html" element and no language is used for the "textarea" element.
+
+SEE ALSO
+
+_oddmu_(1)
+
+"Structuring the web with HTML"
+https://developer.mozilla.org/en-US/docs/Learn/HTML
+
+"Learn to style HTML using CSS"
+https://developer.mozilla.org/en-US/docs/Learn/CSS
+
+The "text/template" library explains how to write templates from a
+programmer perspective. https://pkg.go.dev/text/template
+
+The "html/template" library explains how the templates are made more
+secure in a HTML context. https://pkg.go.dev/html/template
+
+"Lingua" is the library used to detect languages.
+https://github.com/pemistahl/lingua-go
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>. Up-to-date sources can be
+found at https://alexschroeder.ch/cgit/oddmu/.

man/oddmu.1

@@ -0,0 +1,147 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU" "1" "2023-09-22"
+.PP
+.SH NAME
+.PP
+oddmu - a wiki server
+.PP
+Oddmu is sometimes written Oddµ because µ is the letter mu.\&
+.PP
+.SH SYNOPSIS
+.PP
+\fBoddmu\fR
+.PP
+.SH DESCRIPTION
+.PP
+The oddmu program serves the current working directory as a wiki on port 8080.\&
+Point your browser to http://localhost:8080/ to get started.\& This is equivalent
+to http://localhost:8080/view/index – the first page you'\&ll create, most likely.\&
+.PP
+If you request a page that doesn'\&t exist, oddmu tries to find a matching
+Markdown file by appending the extension ".\&md" to the page name.\& In the example
+above, the page name requested is "index" and the file name oddmu tries to read
+is "index.\&md".\& If no such file exists, oddmu offers you to create the page.\&
+.PP
+If your files don'\&t provide their own title ("# title"), the file name (without
+".\&md") is used for the page title.\&
+.PP
+Subdirectories are created as necessary.\&
+.PP
+See \fIoddmu\fR(5) for details about the page formatting.\&
+.PP
+.SH CONFIGURATION
+.PP
+The template files are the HTML files in the working directory: "add.\&html",
+"edit.\&html", "search.\&html", "upload.\&html" and "view.\&html".\& Feel free to change
+the templates and restart the server.\&
+.PP
+The first change you should make is to replace the name and email
+address in the footer of "view.\&html".\& Look for "Your Name" and
+"example.\&org".\&
+.PP
+The second change you should make is to replace the name, email
+address and domain name in "feed.\&html".\& Look for "Your Name" and
+"example.\&org".\& This second template is used to generate the RSS feeds
+(despite it'\&s ".\&html" extension).\&
+.PP
+See \fIoddmu-templates\fR(5) for more.\&
+.PP
+.SH ENVIRONMENT
+.PP
+You can change the port served by setting the ODDMU_PORT environment variable.\&
+.PP
+In order to limit language-detection to the languages you actually use, set the
+environment variable ODDMU_LANGUAGES to a comma-separated list of ISO 639-1
+codes, e.\&g.\& "en" or "en,de,fr,pt".\&
+.PP
+You can enable webfinger to link fediverse accounts to their correct profile
+pages by setting ODDMU_WEBFINGER to "1".\& See \fIoddmu\fR(5).\&
+.PP
+.SH SECURITY
+.PP
+If the machine you are running Oddmu on is accessible from the Internet, you
+must secure your installation.\& The best way to do this is use a regular web
+server as a reverse proxy.\&
+.PP
+See \fIoddmu-apache\fR(5) for an example.\&
+.PP
+.SH OPTIONS
+.PP
+The oddmu program can be run on the command-line using various subcommands.\&
+.PP
+.PD 0
+.IP \(bu 4
+to generate the HTML for a page, see \fIoddmu-html\fR(1)
+.IP \(bu 4
+to search a regular expression and replace it across all files, see
+\fIoddmu-replace\fR(1)
+.IP \(bu 4
+to emulate a search of the files, see \fIoddmu-search\fR(1); to understand how the
+search engine indexes pages and how it sorts and scores results, see
+\fIoddmu-search\fR(7)
+.PD
+.PP
+.SH EXAMPLE
+.PP
+When saving a page, the page name is take from the URL and the page
+content is taken from the "body" form parameter.\& To illustrate, here'\&s
+how to edit a page using \fIcurl\fR(1):
+.PP
+.nf
+.RS 4
+curl --form body="Did you bring a towel?" 
+  http://localhost:8080/save/welcome
+.fi
+.RE
+.PP
+.SH DESIGN
+.PP
+This is a minimal wiki.\& There is no version history.\& It'\&s well suited as a
+\fIsecondary\fR medium: collaboration and conversation happens elsewhere, in chat,
+on social media.\& The wiki serves as the text repository that results from these
+discussions.\&
+.PP
+The wiki lists no recent changes.\& The expectation is that the people that care
+were involved in the discussions beforehand.\&
+.PP
+The wiki also produces no feed.\& The assumption is that announcements are made on
+social media: blogs, news aggregators, discussion forums, the fediverse, but
+humans.\&
+.PP
+The idea is that the webserver handles as many tasks as possible.\& It logs
+requests, does rate limiting, handles encryption, gets the certificates, and so
+on.\& The web server acts as a reverse proxy and the wiki ends up being a content
+management system with almost no structure – or endless malleability, depending
+on your point of view.\& See \fIoddmu-apache\fR(5).\&
+.PP
+.SH NOTES
+.PP
+Page names are filenames with ".\&md" appended.\& If your filesystem cannot handle
+it, it can'\&t be a page name.\&
+.PP
+The pages are indexed as the server starts and the index is kept in memory.\& If
+you have a ton of pages, this takes a lot of memory.\&
+.PP
+Files may not end with a tilde ('\&~'\&) – these are backup files.\&
+.PP
+You cannot edit uploaded files.\& If you upload a file called "hello.\&txt" and
+attempt to edit it by using "/edit/hello.\&txt" you will create a page with the
+name "hello.\&txt.\&md" instead.\&
+.PP
+You cannot delete uploaded files via the web – but you can delete regular wiki
+pages by saving an empty file.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(5), \fIoddmu.\&service\fR(5), oddmu-apache_(5), \fIoddmu-html\fR(1),
+\fIoddmu-replace\fR(1), \fIoddmu-search\fR(1), \fIoddmu-search\fR(7), \fIoddmu-feed\fR(1)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu.1.txt

@@ -0,0 +1,132 @@
+ODDMU(1)
+
+# NAME
+
+oddmu - a wiki server
+
+Oddmu is sometimes written Oddµ because µ is the letter mu.
+
+# SYNOPSIS
+
+*oddmu*
+
+# DESCRIPTION
+
+The oddmu program serves the current working directory as a wiki on port 8080.
+Point your browser to http://localhost:8080/ to get started. This is equivalent
+to http://localhost:8080/view/index – the first page you'll create, most likely.
+
+If you request a page that doesn't exist, oddmu tries to find a matching
+Markdown file by appending the extension ".md" to the page name. In the example
+above, the page name requested is "index" and the file name oddmu tries to read
+is "index.md". If no such file exists, oddmu offers you to create the page.
+
+If your files don't provide their own title ("# title"), the file name (without
+".md") is used for the page title.
+
+Every file can be viewed as feed by using the extension ".rss". The
+feed items are based on links in bullet lists using the asterix ("*").
+
+Subdirectories are created as necessary.
+
+See _oddmu_(5) for details about the page formatting.
+
+# CONFIGURATION
+
+The template files are the HTML files in the working directory: "add.html",
+"edit.html", "search.html", "upload.html" and "view.html". Feel free to change
+the templates and restart the server.
+
+The first change you should make is to replace the name and email
+address in the footer of "view.html". Look for "Your Name" and
+"example.org".
+
+The second change you should make is to replace the name, email
+address and domain name in "feed.html". Look for "Your Name" and
+"example.org". This second template is used to generate the RSS feeds
+(despite it's ".html" extension).
+
+See _oddmu-templates_(5) for more.
+
+# ENVIRONMENT
+
+You can change the port served by setting the ODDMU_PORT environment variable.
+
+In order to limit language-detection to the languages you actually use, set the
+environment variable ODDMU_LANGUAGES to a comma-separated list of ISO 639-1
+codes, e.g. "en" or "en,de,fr,pt".
+
+You can enable webfinger to link fediverse accounts to their correct profile
+pages by setting ODDMU_WEBFINGER to "1". See _oddmu_(5).
+
+# SECURITY
+
+If the machine you are running Oddmu on is accessible from the Internet, you
+must secure your installation. The best way to do this is use a regular web
+server as a reverse proxy.
+
+See _oddmu-apache_(5) for an example.
+
+# OPTIONS
+
+The oddmu program can be run on the command-line using various subcommands.
+
+- to generate the HTML for a page, see _oddmu-html_(1)
+- to search a regular expression and replace it across all files, see
+  _oddmu-replace_(1)
+- to emulate a search of the files, see _oddmu-search_(1); to understand how the
+  search engine indexes pages and how it sorts and scores results, see
+  _oddmu-search_(7)
+
+# EXAMPLE
+
+When saving a page, the page name is take from the URL and the page
+content is taken from the "body" form parameter. To illustrate, here's
+how to edit a page using _curl_(1):
+
+```
+curl --form body="Did you bring a towel?" \
+  http://localhost:8080/save/welcome
+```
+
+# DESIGN
+
+This is a minimal wiki. There is no version history. It's well suited as a
+_secondary_ medium: collaboration and conversation happens elsewhere, in chat,
+on social media. The wiki serves as the text repository that results from these
+discussions.
+
+The wiki lists no recent changes. The expectation is that the people that care
+were involved in the discussions beforehand.
+
+The idea is that the webserver handles as many tasks as possible. It logs
+requests, does rate limiting, handles encryption, gets the certificates, and so
+on. The web server acts as a reverse proxy and the wiki ends up being a content
+management system with almost no structure – or endless malleability, depending
+on your point of view. See _oddmu-apache_(5).
+
+# NOTES
+
+Page names are filenames with ".md" appended. If your filesystem cannot handle
+it, it can't be a page name.
+
+The pages are indexed as the server starts and the index is kept in memory. If
+you have a ton of pages, this takes a lot of memory.
+
+Files may not end with a tilde ('~') – these are backup files.
+
+You cannot edit uploaded files. If you upload a file called "hello.txt" and
+attempt to edit it by using "/edit/hello.txt" you will create a page with the
+name "hello.txt.md" instead.
+
+You cannot delete uploaded files via the web – but you can delete regular wiki
+pages by saving an empty file.
+
+# SEE ALSO
+
+_oddmu_(5), _oddmu.service_(5), oddmu-apache_(5), _oddmu-html_(1),
+_oddmu-replace_(1), _oddmu-search_(1), _oddmu-search_(7)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu.5

@@ -0,0 +1,133 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU" "5" "2023-09-21" "File Formats Manual"
+.PP
+.SH NAME
+.PP
+oddmu - text formatting of wiki pages
+.PP
+.SH SYNTAX
+.PP
+The wiki pages are UTF-8 encoded Markdown files.\&
+.PP
+There are three Oddµ-specific extensions: local links, hashtags and
+fediverse account links.\& The Markdown library used features some
+additional extensions, most importantly tables and definition lists.\&
+.PP
+.SS Local links
+.PP
+Local links use double square brackets [[like this]].\&
+.PP
+.SS Hashtags
+.PP
+Hashtags are single word links to searches for themselves.\& Use the
+underscore to use hashtags consisting of multiple words.\& Hashtags are
+distinguished from page titles because there is no space after the
+hash.\&
+.PP
+.nf
+.RS 4
+# Title
+
+Text
+
+#Tag #Another_Tag
+.fi
+.RE
+.PP
+.SS Tables
+.PP
+A table with footers and a columnspan:
+.PP
+.nf
+.RS 4
+Name    | Age
+--------|------
+Bob     ||
+Alice   | 23
+========|======
+Total   | 23
+.fi
+.RE
+.PP
+.SS Definition lists:
+.PP
+.nf
+.RS 4
+Cat
+: Fluffy animal everyone likes
+
+Internet
+: Vector of transmission for pictures of cats
+.fi
+.RE
+.PP
+.SS Fediverse account links
+.PP
+Fediverse accounts look a bit like an at sign followed by an email
+address, e.\&g.\& @alex@alexschroeder.\&ch.\& When rendering a page, these
+turn into a username linked to a profile page.\& In this case, "@alex"
+would be linked to "https://alexschroeder.\&ch/users/alex".\&
+.PP
+In many cases, this will work as is.\& In reality, however, the link to
+the profile page needs to be retrieved via webfinger.\& Oddµ does that
+in the background, and as soon as the information is available, the
+actual profile link is used when pages are rendered.\& In the example
+above, the result would be "https://social.\&alexschroeder.\&ch/@alex".\&
+.PP
+As this sort of packground network activity is surprising, it is not
+enabled by default.\& Set the environment variable ODDMU_WEBFINGER to
+"1" in order to enable this.\&
+.PP
+.SS Other extensions
+.PP
+The Markdown processor comes with a few extensions:
+.PP
+\fB emphasis markers inside words are ignored
+\fR fenced code blocks are supported
+\fB autolinking of "naked" URLs are supported
+\fR strikethrough using two tildes is supported (~~like this~~)
+\fB it is strict about prefix heading rules
+\fR you can specify an id for headings ({#id})
+\fB trailing backslashes turn into line breaks
+\fR MathJax is supported (but needs a separte setup)
+.PP
+.SH PERCENT ENCODING
+.PP
+If you use Markdown links to local pages, you must percent-encode the
+link target.\& Any character that is not an "unreserved character"
+according to RFC 3986 might need to be encoded.\& The unreserved
+characters are a-z, A-Z, 0-9, as well as the four characters '\&-'\&,
+\&'\&_'\&, '\&.\&'\& and '\&~'\&.\&
+.PP
+Percent-encoding means that each character is converted into one or
+more bytes, and each byte is represented as a percent character
+followed by a hexadecimal representation.\&
+.PP
+Realistically, what probably works best is to use a browser.\& If you
+type "http://example.\&org/Alex Schröder" into the address bar, you'\&ll
+get sent to the example domain.\& If you now copy the address and paste
+it back into a text editor, you'\&ll get
+"http://example.\&org/Alex%20Schr%C3%B6der" and that'\&s how you'\&ll learn
+that the Space is encoded by %20 and that the character '\&ö'\& is encoded
+by %C3%B6.\& To link to the page "Alex Schröder" you would write
+something like this: "[Alex](Alex%20Schr%C3%B6der)".\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1)
+.PP
+This wiki uses the Go Markdown library.\&
+https://github.\&com/gomarkdown/markdown
+.PP
+For more about percent-encoding, see Wikipedia.\&
+https://en.\&wikipedia.\&org/wiki/Percent-encoding
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu.5.txt

@@ -0,0 +1,149 @@
+ODDMU(5) "File Formats Manual"
+
+# NAME
+
+oddmu - text formatting of wiki pages
+
+# SYNTAX
+
+The wiki pages are UTF-8 encoded Markdown files.
+
+There are three Oddµ-specific extensions: local links, hashtags and
+fediverse account links. The Markdown library used features some
+additional extensions, most importantly tables and definition lists.
+
+## Local links
+
+Local links use double square brackets [[like this]].
+
+## Hashtags
+
+Hashtags are single word links to searches for themselves. Use the
+underscore to use hashtags consisting of multiple words. Hashtags are
+distinguished from page titles because there is no space after the
+hash.
+
+```
+# Title
+
+Text
+
+#Tag #Another_Tag
+```
+
+## Tables
+
+A table with footers and a columnspan:
+
+```
+Name    | Age
+--------|------
+Bob     ||
+Alice   | 23
+========|======
+Total   | 23
+```
+
+## Definition lists:
+
+```
+Cat
+: Fluffy animal everyone likes
+
+Internet
+: Vector of transmission for pictures of cats
+```
+
+## Fediverse account links
+
+Fediverse accounts look a bit like an at sign followed by an email
+address, e.g. @alex@alexschroeder.ch. When rendering a page, these
+turn into a username linked to a profile page. In this case, "@alex"
+would be linked to "https://alexschroeder.ch/users/alex".
+
+In many cases, this will work as is. In reality, however, the link to
+the profile page needs to be retrieved via webfinger. Oddµ does that
+in the background, and as soon as the information is available, the
+actual profile link is used when pages are rendered. In the example
+above, the result would be "https://social.alexschroeder.ch/@alex".
+
+As this sort of packground network activity is surprising, it is not
+enabled by default. Set the environment variable ODDMU_WEBFINGER to
+"1" in order to enable this.
+
+## Other extensions
+
+The Markdown processor comes with a few extensions:
+
+* emphasis markers inside words are ignored
+* fenced code blocks are supported
+* autolinking of "naked" URLs are supported
+* strikethrough using two tildes is supported (~~like this~~)
+* it is strict about prefix heading rules
+* you can specify an id for headings ({#id})
+* trailing backslashes turn into line breaks
+* MathJax is supported (but needs a separte setup)
+
+# FEEDS
+
+Every file can be viewed as feed by using the extension ".rss". The
+feed items are based on links in bullet lists using the asterix ("*").
+The items must point to local pages. This is why the link may not
+contain two forward slashes ("//").
+
+Assume this is the index page. The feed would be "/view/index.rss". It
+would contain the pages "Arianism", "Donatism" and "Monophysitism" but
+it would not contain the pages "Feed" and "About" since the list items
+don't start with an asterix.
+
+```
+# Main Page
+
+Hello and welcome! Here are some important links:
+
+- [Feed](index.rss)
+- [About](about)
+
+Recent posts:
+
+* [Arianism](arianism)
+* [Donatism](donatism)
+* [Monophysitism](monophysitism)
+```
+
+The feed contains at most 10 items, starting at the top.
+
+# PERCENT ENCODING
+
+If you use Markdown links to local pages, you must percent-encode the
+link target. Any character that is not an "unreserved character"
+according to RFC 3986 might need to be encoded. The unreserved
+characters are a-z, A-Z, 0-9, as well as the four characters '-',
+'\_', '.' and '~'.
+
+Percent-encoding means that each character is converted into one or
+more bytes, and each byte is represented as a percent character
+followed by a hexadecimal representation.
+
+Realistically, what probably works best is to use a browser. If you
+type "http://example.org/Alex Schröder" into the address bar, you'll
+get sent to the example domain. If you now copy the address and paste
+it back into a text editor, you'll get
+"http://example.org/Alex%20Schr%C3%B6der" and that's how you'll learn
+that the Space is encoded by %20 and that the character 'ö' is encoded
+by %C3%B6. To link to the page "Alex Schröder" you would write
+something like this: "[Alex](Alex%20Schr%C3%B6der)".
+
+# SEE ALSO
+
+_oddmu_(1)
+
+This wiki uses the Go Markdown library.
+https://github.com/gomarkdown/markdown
+
+For more about percent-encoding, see Wikipedia.
+https://en.wikipedia.org/wiki/Percent-encoding
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu.service.5

@@ -0,0 +1,82 @@
+.\" Generated by scdoc 1.11.2
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU.SERVICE" "5" "2023-09-21"
+.PP
+.SH NAME
+.PP
+oddmu.\&service - how to setup Oddmu using systemd
+.PP
+.SS DESCRIPTION
+.PP
+Here'\&s how to setup a wiki using systemd such that it starts
+automatically when the system boots and gets restarted automatically
+when it crashes.\&
+.PP
+First, create a new user called "oddmu" with it'\&s own home directory
+but without a login.\&
+.PP
+.nf
+.RS 4
+adduser --system --home /home/oddmu oddmu
+.fi
+.RE
+.PP
+The directory "/home/oddmu" contains the templates and all the data
+files.\& Copy all the tempaltes files ending in ".\&html" from the source
+distribution to "/home/oddmu".\&
+.PP
+If you want to keep everything in one place, copy the binary "oddmu"
+and the service file "oddmu.\&service" to "/home/oddmu", too.\&
+.PP
+Edit the `oddmu.\&service` file.\& These are the lines you most likely
+have to take care of:
+.PP
+.nf
+.RS 4
+ExecStart=/home/oddmu/oddmu
+WorkingDirectory=/home/oddmu
+Environment="ODDMU_PORT=8080"
+Environment="ODDMU_WEBFINGER=1"
+.fi
+.RE
+.PP
+Install the service file and enable it:
+.PP
+.nf
+.RS 4
+ln -s /home/oddmu/oddmu\&.service /etc/systemd/system/
+systemctl enable --now oddmu
+.fi
+.RE
+.PP
+You should be able to visit the wiki at
+http://localhost:8080/.\&
+.PP
+Check the log:
+.PP
+.nf
+.RS 4
+journalctl --unit oddmu
+.fi
+.RE
+.PP
+Follow the log:
+.PP
+.nf
+.RS 4
+journalctl --follow --unit oddmu
+.fi
+.RE
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1), \fIsystemd.\&exec\fR(5), \fIcapabilities\fR(7)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu.service.5.txt

@@ -0,0 +1,65 @@
+ODDMU.SERVICE(5)
+
+# NAME
+
+oddmu.service - how to setup Oddmu using systemd
+
+## DESCRIPTION
+
+Here's how to setup a wiki using systemd such that it starts
+automatically when the system boots and gets restarted automatically
+when it crashes.
+
+First, create a new user called "oddmu" with it's own home directory
+but without a login.
+
+```
+adduser --system --home /home/oddmu oddmu
+```
+
+The directory "/home/oddmu" contains the templates and all the data
+files. Copy all the tempaltes files ending in ".html" from the source
+distribution to "/home/oddmu".
+
+If you want to keep everything in one place, copy the binary "oddmu"
+and the service file "oddmu.service" to "/home/oddmu", too.
+
+Edit the `oddmu.service` file. These are the lines you most likely
+have to take care of:
+
+```
+ExecStart=/home/oddmu/oddmu
+WorkingDirectory=/home/oddmu
+Environment="ODDMU_PORT=8080"
+Environment="ODDMU_WEBFINGER=1"
+```
+
+Install the service file and enable it:
+
+```
+ln -s /home/oddmu/oddmu.service /etc/systemd/system/
+systemctl enable --now oddmu
+```
+
+You should be able to visit the wiki at
+http://localhost:8080/.
+
+Check the log:
+
+```
+journalctl --unit oddmu
+```
+
+Follow the log:
+
+```
+journalctl --follow --unit oddmu
+```
+
+# SEE ALSO
+
+_oddmu_(1), _systemd.exec_(5), _capabilities_(7)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

oddmu.service

@@ -12,6 +12,7 @@ MemoryHigh=120M
 ExecStart=/home/oddmu/oddmu
 WorkingDirectory=/home/oddmu
 Environment="ODDMU_PORT=8080"
+Environment="ODDMU_WEBFINGER=1"
 
 # (man "systemd.exec")
 ReadWritePaths=/home/oddmu
@@ -21,7 +22,6 @@ RemoveIPC=yes
 MemoryDenyWriteExecute=yes
 
 # Sandboxing options to harden security
-# Details for these options: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
 NoNewPrivileges=yes
 PrivateTmp=yes
 PrivateDevices=yes
@@ -37,7 +37,7 @@ LockPersonality=yes
 SystemCallFilter=~@clock @debug @module @mount @obsolete @reboot @setuid @swap
 
 # Denying access to capabilities that should not be relevant
-# Doc: https://man7.org/linux/man-pages/man7/capabilities.7.html
+# (man "capabilities")
 CapabilityBoundingSet=~CAP_RAWIO CAP_MKNOD
 CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE
 CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_TIME CAP_SYS_MODULE CAP_SYS_PACCT

page.go

@@ -3,9 +3,6 @@ package main
 import (
 	"bytes"
 	"fmt"
-	"github.com/gomarkdown/markdown"
-	"github.com/gomarkdown/markdown/ast"
-	"github.com/gomarkdown/markdown/parser"
 	"github.com/microcosm-cc/bluemonday"
 	"html/template"
 	"net/url"
@@ -27,16 +24,20 @@ type Page struct {
 	Body     []byte
 	Html     template.HTML
 	Score    int
+	Hashtags []string
 }
 
 // santize uses bluemonday to sanitize the HTML.
-func sanitize(s string) template.HTML {
-	return template.HTML(bluemonday.UGCPolicy().Sanitize(s))
+// No exceptions are made because this is used for snippets.
+func sanitizeStrict(s string) template.HTML {
+	return template.HTML(bluemonday.StrictPolicy().Sanitize(s))
 }
 
 // santizeBytes uses bluemonday to sanitize the HTML.
 func sanitizeBytes(bytes []byte) template.HTML {
-	return template.HTML(bluemonday.UGCPolicy().SanitizeBytes(bytes))
+	policy := bluemonday.UGCPolicy()
+	policy.AllowAttrs("class").OnElements("a") // for hashtags
+	return template.HTML(policy.SanitizeBytes(bytes))
 }
 
 // nameEscape returns the page name safe for use in URLs. That is,
@@ -59,7 +60,7 @@ func (p *Page) save() error {
 	filename := p.Name + ".md"
 	s := bytes.ReplaceAll(p.Body, []byte{'\r'}, []byte{})
 	if len(s) == 0 {
-		_ = os.Rename(filename, filename + "~")
+		_ = os.Rename(filename, filename+"~")
 		return os.Remove(filename)
 	}
 	p.Body = s
@@ -72,7 +73,7 @@ func (p *Page) save() error {
 			return err
 		}
 	}
-	_ = os.Rename(filename, filename + "~")
+	_ = os.Rename(filename, filename+"~")
 	return os.WriteFile(filename, s, 0644)
 }
 
@@ -104,68 +105,17 @@ func (p *Page) handleTitle(replace bool) {
 	}
 }
 
-func hashtag(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
-	data = data[offset:]
-	i := 0
-	n := len(data)
-	for i < n && !parser.IsSpace(data[i]) {
-		i++
-	}
-	if i == 0 {
-		return 0, nil
-	}
-	link := &ast.Link{
-		Destination: append([]byte("/search?q=%23"), data[1:i]...),
-		Title:       data[0:i],
-	}
-	text := bytes.ReplaceAll(data[0:i], []byte("_"), []byte(" "))
-	ast.AppendChild(link, &ast.Text{Leaf: ast.Leaf{Literal: text}})
-	return i, link
-}
-
-// renderHtml renders the Page.Body to HTML and sets Page.Html.
-func (p *Page) renderHtml() {
-	parser := parser.New()
-	parser.RegisterInline('#', hashtag)
-	maybeUnsafeHTML := markdown.ToHTML(p.Body, parser, nil)
-	p.Name = nameEscape(p.Name)
-	p.Html = sanitizeBytes(maybeUnsafeHTML)
-	p.Language = language(p.plainText())
-}
-
-// plainText renders the Page.Body to plain text and returns it,
-// ignoring all the Markdown and all the newlines. The result is one
-// long single line of text.
-func (p *Page) plainText() string {
-	parser := parser.New()
-	doc := markdown.Parse(p.Body, parser)
-	text := []byte("")
-	ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
-		if entering && node.AsLeaf() != nil {
-			text = append(text, node.AsLeaf().Literal...)
-			text = append(text, []byte(" ")...)
-		}
-		return ast.GoToNext
-	})
-	// Some Markdown still contains newlines
-	for i, c := range text {
-		if c == '\n' {
-			text[i] = ' '
-		}
-	}
-	// Remove trailing space
-	for len(text) > 0 && text[len(text)-1] == ' ' {
-		text = text[0 : len(text)-1]
-	}
-	return string(text)
-}
-
-// summarize for query string q sets Page.Html to an extract.
-func (p *Page) summarize(q string) {
+// score sets Page.Title and computes Page.Score.
+func (p *Page) score(q string) {
 	p.handleTitle(true)
 	p.Score = score(q, string(p.Body)) + score(q, p.Title)
+}
+
+// summarize sets Page.Html to an extract and sets Page.Language.
+func (p *Page) summarize(q string) {
 	t := p.plainText()
-	p.Html = sanitize(snippets(q, t))
+	p.Name = nameEscape(p.Name)
+	p.Html = sanitizeStrict(snippets(q, t))
 	p.Language = language(t)
 }
 

page_test.go

@@ -19,49 +19,6 @@ But yearn for blue sky`)}
 	assert.Regexp(t, regexp.MustCompile("^My back"), string(p.Body))
 }
 
-func TestPagePlainText(t *testing.T) {
-	p := &Page{Body: []byte(`# Water
-The air will not come
-To inhale is an effort
-The summer heat kills`)}
-	r := "Water The air will not come To inhale is an effort The summer heat kills"
-	assert.Equal(t, r, p.plainText())
-}
-
-func TestPageHtml(t *testing.T) {
-	p := &Page{Body: []byte(`# Sun
-Silver leaves shine bright
-They droop, boneless, weak and sad
-A cruel sun stares down`)}
-	p.renderHtml()
-	r := `<h1>Sun</h1>
-
-<p>Silver leaves shine bright
-They droop, boneless, weak and sad
-A cruel sun stares down</p>
-`
-	assert.Equal(t, r, string(p.Html))
-}
-
-func TestPageHtmlHashtag(t *testing.T) {
-	p := &Page{Body: []byte(`# Comet
-Stars flicker above
-Too faint to focus, so far
-I am cold, alone
-
-#Haiku #Cold_Poets`)}
-	p.renderHtml()
-	r := `<h1>Comet</h1>
-
-<p>Stars flicker above
-Too faint to focus, so far
-I am cold, alone</p>
-
-<p><a href="/search?q=%23Haiku" rel="nofollow">#Haiku</a> <a href="/search?q=%23Cold_Poets" rel="nofollow">#Cold Poets</a></p>
-`
-	assert.Equal(t, r, string(p.Html))
-}
-
 // wipes testdata
 func TestPageDir(t *testing.T) {
 	_ = os.RemoveAll("testdata")
@@ -71,7 +28,7 @@ From bed to bathroom
 A slow shuffle in the dark
 Moonlight floods the aisle`)}
 	p.save()
-	
+
 	o, err := loadPage("testdata/moon")
 	assert.NoError(t, err, "load page")
 	assert.Equal(t, p.Body, o.Body)
@@ -84,7 +41,7 @@ Moonlight floods the aisle`)}
 
 	// But the backup still exists.
 	assert.FileExists(t, "testdata/moon.md~")
-	
+
 	t.Cleanup(func() {
 		_ = os.RemoveAll("testdata")
 	})

parser.go

@@ -0,0 +1,104 @@
+package main
+
+import (
+	"bytes"
+	"github.com/gomarkdown/markdown"
+	"github.com/gomarkdown/markdown/ast"
+	"github.com/gomarkdown/markdown/parser"
+	"net/url"
+)
+
+// wikiLink returns an inline parser function. This indirection is
+// required because we want to call the previous definition in case
+// this is not a wikiLink.
+func wikiLink(p *parser.Parser, fn func(p *parser.Parser, data []byte, offset int) (int, ast.Node)) func(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
+	return func(p *parser.Parser, original []byte, offset int) (int, ast.Node) {
+		data := original[offset:]
+		n := len(data)
+		// minimum: [[X]]
+		if n < 5 || data[1] != '[' {
+			return fn(p, original, offset)
+		}
+		i := 2
+		for i+1 < n && data[i] != ']' && data[i+1] != ']' {
+			i++
+		}
+		text := data[2 : i+1]
+		link := &ast.Link{
+			Destination: []byte(url.PathEscape(string(text))),
+		}
+		ast.AppendChild(link, &ast.Text{Leaf: ast.Leaf{Literal: text}})
+		return i + 3, link
+	}
+}
+
+// hashtag returns an inline parser function. This indirection is
+// required because we want to receive an array of hashtags found.
+func hashtag() (func(p *parser.Parser, data []byte, offset int) (int, ast.Node), *[]string) {
+	hashtags := make([]string,0)
+	return func(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
+		data = data[offset:]
+		i := 0
+		n := len(data)
+		for i < n && !parser.IsSpace(data[i]) {
+			i++
+		}
+		if i == 0 {
+			return 0, nil
+		}
+		hashtags = append(hashtags, string(data[1:i]))
+		link := &ast.Link{
+			AdditionalAttributes: []string{`class="tag"`},
+			Destination: append([]byte("/search?q=%23"), data[1:i]...),
+			Title:       data[0:i],
+		}
+		text := bytes.ReplaceAll(data[0:i], []byte("_"), []byte(" "))
+		ast.AppendChild(link, &ast.Text{Leaf: ast.Leaf{Literal: text}})
+		return i, link
+	}, &hashtags
+}
+
+// renderHtml renders the Page.Body to HTML and sets Page.Html,
+// Page.Language, Page.Hashtags, and escapes Page.Name.
+func (p *Page) renderHtml() {
+	parser := parser.New()
+	prev := parser.RegisterInline('[', nil)
+	parser.RegisterInline('[', wikiLink(parser, prev))
+	fn, hashtags := hashtag()
+	parser.RegisterInline('#', fn)
+	if useWebfinger {
+		parser.RegisterInline('@', account)
+	}
+	maybeUnsafeHTML := markdown.ToHTML(p.Body, parser, nil)
+	p.Name = nameEscape(p.Name)
+	p.Html = sanitizeBytes(maybeUnsafeHTML)
+	p.Language = language(p.plainText())
+	p.Hashtags = *hashtags
+}
+
+// plainText renders the Page.Body to plain text and returns it,
+// ignoring all the Markdown and all the newlines. The result is one
+// long single line of text.
+func (p *Page) plainText() string {
+	parser := parser.New()
+	doc := markdown.Parse(p.Body, parser)
+	text := []byte("")
+	ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
+		if entering && node.AsLeaf() != nil {
+			text = append(text, node.AsLeaf().Literal...)
+			text = append(text, []byte(" ")...)
+		}
+		return ast.GoToNext
+	})
+	// Some Markdown still contains newlines
+	for i, c := range text {
+		if c == '\n' {
+			text[i] = ' '
+		}
+	}
+	// Remove trailing space
+	for len(text) > 0 && text[len(text)-1] == ' ' {
+		text = text[0 : len(text)-1]
+	}
+	return string(text)
+}

parser_test.go

@@ -0,0 +1,64 @@
+package main
+
+import (
+	"github.com/stretchr/testify/assert"
+	"testing"
+)
+
+func TestPagePlainText(t *testing.T) {
+	p := &Page{Body: []byte(`# Water
+The air will not come
+To inhale is an effort
+The summer heat kills`)}
+	r := "Water The air will not come To inhale is an effort The summer heat kills"
+	assert.Equal(t, r, p.plainText())
+}
+
+func TestPageHtml(t *testing.T) {
+	p := &Page{Body: []byte(`# Sun
+Silver leaves shine bright
+They droop, boneless, weak and sad
+A cruel sun stares down`)}
+	p.renderHtml()
+	r := `<h1>Sun</h1>
+
+<p>Silver leaves shine bright
+They droop, boneless, weak and sad
+A cruel sun stares down</p>
+`
+	assert.Equal(t, r, string(p.Html))
+}
+
+func TestPageHtmlHashtag(t *testing.T) {
+	p := &Page{Body: []byte(`# Comet
+Stars flicker above
+Too faint to focus, so far
+I am cold, alone
+
+#Haiku #Cold_Poets`)}
+	p.renderHtml()
+	r := `<h1>Comet</h1>
+
+<p>Stars flicker above
+Too faint to focus, so far
+I am cold, alone</p>
+
+<p><a class="tag" href="/search?q=%23Haiku" rel="nofollow">#Haiku</a> <a class="tag" href="/search?q=%23Cold_Poets" rel="nofollow">#Cold Poets</a></p>
+`
+	assert.Equal(t, r, string(p.Html))
+}
+
+func TestPageHtmlWikiLink(t *testing.T) {
+	p := &Page{Body: []byte(`# Photos and Books
+Blue and green and black
+Sky and grass and [ragged cliffs](cliffs)
+Our [[time together]]`)}
+	p.renderHtml()
+	r := `<h1>Photos and Books</h1>
+
+<p>Blue and green and black
+Sky and grass and <a href="cliffs" rel="nofollow">ragged cliffs</a>
+Our <a href="time%20together" rel="nofollow">time together</a></p>
+`
+	assert.Equal(t, r, string(p.Html))
+}

replace_cmd.go

@@ -0,0 +1,92 @@
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"github.com/google/subcommands"
+	"github.com/hexops/gotextdiff"
+	"github.com/hexops/gotextdiff/myers"
+	"github.com/hexops/gotextdiff/span"
+	"io"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"regexp"
+	"slices"
+	"strings"
+)
+
+type replaceCmd struct {
+	confirm bool
+}
+
+func (cmd *replaceCmd) SetFlags(f *flag.FlagSet) {
+	f.BoolVar(&cmd.confirm, "confirm", false, "do the replacement instead of just doing a dry run")
+}
+
+func (*replaceCmd) Name() string     { return "replace" }
+func (*replaceCmd) Synopsis() string { return "Search and replace a regular expression." }
+func (*replaceCmd) Usage() string {
+	return `replace [-confirm] <regexp> <replacement>:
+  Search a regular expression and replace it. By default, this is a
+  dry run and nothing is saved. The replacement can use $1, $2, etc.
+  to refer to capture groups in the regular expression.
+`
+}
+
+func (cmd *replaceCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+	return replaceCli(os.Stdout, cmd.confirm, f.Args())
+}
+
+func replaceCli(w io.Writer, confirm bool, args []string) subcommands.ExitStatus {
+	if len(args) != 2 {
+		fmt.Fprintln(w, "Replace takes exactly two arguments.")
+		return subcommands.ExitFailure
+	}
+	re := regexp.MustCompile(args[0])
+	repl := []byte(args[1])
+	changes := 0
+	err := filepath.Walk(".", func(path string, info fs.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		if info.IsDir() || strings.HasPrefix(path, ".") || !strings.HasSuffix(path, ".md") {
+			return nil
+		}
+		body, err := os.ReadFile(path)
+		if err != nil {
+			return err
+		}
+		result := re.ReplaceAll(body, repl)
+		if !slices.Equal(result, body) {
+			changes++
+			if confirm {
+				fmt.Fprintln(w, path)
+				_ = os.Rename(path, path+"~")
+				err = os.WriteFile(path, result, 0644)
+				if err != nil {
+					return err
+				}
+			} else {
+				edits := myers.ComputeEdits(span.URIFromPath(path+"~"), string(body), string(result))
+				diff := fmt.Sprint(gotextdiff.ToUnified(path+"~", path, string(body), edits))
+				fmt.Fprintln(w, diff)
+			}
+		}
+		return nil
+	})
+	if err != nil {
+		fmt.Fprintln(w, err)
+		return subcommands.ExitFailure
+	}
+	if changes == 1 {
+		fmt.Fprintln(w, "1 change was made.")
+	} else {
+		fmt.Fprintf(w, "%d changes were made.\n", changes)
+	}
+	if !confirm && changes > 0 {
+		fmt.Fprintln(w, "This is a dry run. Use -confirm to make it happen.")
+	}
+	return subcommands.ExitSuccess
+}

replace_cmd_test.go

@@ -0,0 +1,44 @@
+package main
+
+import (
+	"bytes"
+	"github.com/stretchr/testify/assert"
+	"github.com/google/subcommands"
+	"os"
+	"testing"
+)
+
+// wipes testdata
+func TestReplaceCmd(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	index.load()
+	p := &Page{Name: "testdata/pluto", Body: []byte(`# Pluto
+Out there is a rock
+And more rocks uncountable
+You are no planet`)}
+	p.save()
+
+	r := `--- testdata/pluto.md~
++++ testdata/pluto.md
+@@ -1,4 +1,4 @@
+ # Pluto
+ Out there is a rock
+ And more rocks uncountable
+-You are no planet
+\ No newline at end of file
++You are planetoid
+\ No newline at end of file
+
+1 change was made.
+This is a dry run. Use -confirm to make it happen.
+`
+	
+	b := new(bytes.Buffer)
+	s := replaceCli(b, false, []string{`\bno planet`, `planetoid`})
+	assert.Equal(t, subcommands.ExitSuccess, s)
+	assert.Equal(t, r, b.String())
+
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}

score_test.go

@@ -94,7 +94,7 @@ func TestScorePageAndMarkup(t *testing.T) {
 	s := `The Transjovian Council accepts new members. If you think we'd be a good fit, apply for an account. Contact [Alex Schroeder](https://alexschroeder.ch/wiki/Contact). Mail is best. Encrypted mail is best. [Delta Chat](https://delta.chat/de/) is a messenger app that uses encrypted mail. It's the bestest best.`
 	p := &Page{Title: "Test", Name: "Test", Body: []byte(s)}
 	q := "wiki"
-	p.summarize(q)
+	p.score(q)
 	// "wiki" is not visible in the plain text but the score is no affected:
 	// - wiki, all, whole, beginning, end (5)
 	if p.Score != 5 {

search.go

@@ -4,6 +4,8 @@ import (
 	"fmt"
 	"net/http"
 	"slices"
+	"strconv"
+	"strings"
 	"unicode"
 	"unicode/utf8"
 )
@@ -13,68 +15,98 @@ import (
 // Currently there is no pagination of results! When a page is part of
 // a search result, Body and Html are simple extracts.
 type Search struct {
-	Query   string
-	Items   []Page
-	Results bool
+	Query    string
+	Items    []*Page
+	Previous int
+	Page     int
+	Next     int
+	Last     int
+	More     bool
+	Results  bool
 }
 
-func sortItems(a, b Page) int {
-	// Sort by score
-	if a.Score < b.Score {
-		return 1
-	} else if a.Score > b.Score {
-		return -1
-	}
-	// If the score is the same and both page names start
-	// with a number (like an ISO date), sort descending.
-	ra, _ := utf8.DecodeRuneInString(a.Title)
-	rb, _ := utf8.DecodeRuneInString(b.Title)
-	if unicode.IsNumber(ra) && unicode.IsNumber(rb) {
-		if a.Title < b.Title {
-			return 1
-		} else if a.Title > b.Title {
+// sortNames returns a sort function that sorts in three stages: 1.
+// whether the query string matches the page title; 2. descending if
+// the page titles start with a digit; 3. otherwise ascending.
+// Access to the index requires a read lock!
+func sortNames(q string) func(a, b string) int {
+	return func(a, b string) int {
+		// If only one page contains the query string, it
+		// takes precedence.
+		ia := strings.Contains(index.titles[a], q)
+		ib := strings.Contains(index.titles[b], q)
+		if ia && !ib {
 			return -1
+		} else if !ia && ib {
+			return 1
+		}
+		// If both page names start with a number (like an ISO date),
+		// sort descending.
+		ra, _ := utf8.DecodeRuneInString(a)
+		rb, _ := utf8.DecodeRuneInString(b)
+		if unicode.IsNumber(ra) && unicode.IsNumber(rb) {
+			if a < b {
+				return 1
+			} else if a > b {
+				return -1
+			} else {
+				return 0
+			}
+		}
+		// Otherwise sort by title, ascending.
+		if index.titles[a] < index.titles[b] {
+			return -1
+		} else if index.titles[a] > index.titles[b] {
+			return 1
 		} else {
 			return 0
 		}
 	}
-	// Otherwise sort ascending.
-	if a.Title < b.Title {
-		return -1
-	} else if a.Title > b.Title {
-		return 1
-	} else {
-		return 0
-	}
 }
 
-// loadAndSummarize loads the pages named and summarizes them for the
-// query give.
-func loadAndSummarize(names []string, q string) []Page {
-	// Load and summarize the items.
-	items := make([]Page, len(names))
+// load the pages named.
+func load(names []string) []*Page {
+	items := make([]*Page, len(names))
 	for i, name := range names {
 		p, err := loadPage(name)
 		if err != nil {
 			fmt.Printf("Error loading %s\n", name)
 		} else {
-			p.summarize(q)
-			items[i] = *p
+			items[i] = p
 		}
 	}
 	return items
 }
 
+// itemsPerPage says how many items to print on a page of search
+// results.
+const itemsPerPage = 20
+
 // search returns a sorted []Page where each page contains an extract
-// of the actual Page.Body in its Page.Html.
-func search(q string) []Page {
+// of the actual Page.Body in its Page.Html. Page size is 20. The
+// boolean return value indicates whether there are more results.
+func search(q string, page int) ([]*Page, bool, int) {
 	if len(q) == 0 {
-		return make([]Page, 0)
+		return make([]*Page, 0), false, 0
 	}
+	index.RLock()
 	names := searchDocuments(q)
-	items := loadAndSummarize(names, q)
-	slices.SortFunc(items, sortItems)
-	return items
+	slices.SortFunc(names, sortNames(q))
+	index.RUnlock()
+	from := itemsPerPage * (page - 1)
+	if from > len(names) {
+		return make([]*Page, 0), false, 0
+	}
+	to := from + itemsPerPage
+	if to > len(names) {
+		to = len(names)
+	}
+	items := load(names[from:to])
+	for _, p := range items {
+		p.score(q)
+		p.summarize(q)
+	}
+	return items, to < len(names), len(names)/itemsPerPage + 1
 }
 
 // searchHandler presents a search result. It uses the query string in
@@ -82,7 +114,12 @@ func search(q string) []Page {
 // page found, the HTML is just an extract of the actual body.
 func searchHandler(w http.ResponseWriter, r *http.Request) {
 	q := r.FormValue("q")
-	items := search(q)
-	s := &Search{Query: q, Items: items, Results: len(items) > 0}
+	page, err := strconv.Atoi(r.FormValue("page"))
+	if err != nil {
+		page = 1
+	}
+	items, more, last := search(q, page)
+	s := &Search{Query: q, Items: items, Previous: page - 1, Page: page, Next: page + 1, Last: last,
+		Results: len(items) > 0, More: more}
 	renderTemplate(w, "search", s)
 }

search.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
     <meta charset="utf-8">
     <meta name="format-detection" content="telephone=no">
@@ -10,23 +10,32 @@ html { max-width: 65ch; padding: 2ch; margin: auto; color: #111; background: #ff
 body { hyphens: auto; }
 header a { margin-right: 1ch; }
 form { display: inline-block; }
+input#search { width: 20ch; }
+button { background: #eee; color: inherit; border-radius: 4px; border-width: 1px; }
 img { max-width: 20%; }
 .result { font-size: larger }
 .score { font-size: smaller; opacity: 0.8; }
     </style>
   </head>
-  <body lang="en">
+  <body>
     <header>
       <a href="#main">Skip navigation</a>
       <a href="/view/index">Home</a>
       <form role="search" action="/search" method="GET">
-        <input type="text" value="{{.Query}}" spellcheck="false" name="q" required>
-        <button>Search</button>
+        <label for="search">Search:</label>
+        <input id="search" type="text" value="{{.Query}}" spellcheck="false" name="q" accesskey="f" required>
+        <button>Go</button>
       </form>
     </header>
     <main id="main">
       <h1>Search for {{.Query}}</h1>
       {{if .Results}}
+      <p>
+        {{if gt .Page 2}}<a href="/search?q={{.Query}}&page=1">First</a>{{end}}
+        {{if gt .Page 1}}<a href="/search?q={{.Query}}&page={{.Previous}}">Previous</a>{{end}}
+        Page {{.Page}}
+        {{if .More}}<a href="/search?q={{.Query}}&page={{.Next}}">Next</a>{{end}}
+        {{if lt .Next .Last}}<a href="/search?q={{.Query}}&page={{.Last}}">Last</a>{{end}}
       {{range .Items}}
       <article lang="{{.Language}}">
         <p><a class="result" href="/view/{{.Name}}">{{.Title}}</a>
@@ -34,6 +43,12 @@ img { max-width: 20%; }
         <blockquote>{{.Html}}</blockquote>
       </article>
       {{end}}
+      <p>
+        {{if gt .Page 2}}<a href="/search?q={{.Query}}&page=1">First</a>{{end}}
+        {{if gt .Page 1}}<a href="/search?q={{.Query}}&page={{.Previous}}">Previous</a>{{end}}
+        Page {{.Page}}
+        {{if .More}}<a href="/search?q={{.Query}}&page={{.Next}}">Next</a>{{end}}
+        {{if lt .Next .Last}}<a href="/search?q={{.Query}}&page={{.Last}}">Last</a>{{end}}
       {{else}}
       <p>No results.</p>
       {{end}}

search_cmd.go

@@ -0,0 +1,54 @@
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"github.com/google/subcommands"
+	"io"
+	"os"
+)
+
+type searchCmd struct {
+	page int
+}
+
+func (cmd *searchCmd) SetFlags(f *flag.FlagSet) {
+	f.IntVar(&cmd.page, "page", 1, "the page in the search result set")
+}
+
+func (*searchCmd) Name() string     { return "search" }
+func (*searchCmd) Synopsis() string { return "Search pages and print a list of links." }
+func (*searchCmd) Usage() string {
+	return `search [-page <n>] <terms>:
+  Search for pages matching terms and print the result set as a
+  Markdown list. Before searching, all the pages are indexed. Thus,
+  startup is slow. The benefit is that the page order and scores are
+  exactly as when the wiki runs.
+`
+}
+
+func (cmd *searchCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+	return searchCli(os.Stdout, cmd.page, f.Args())
+}
+
+// searchCli runs the search command on the command line. It is used
+// here with an io.Writer for easy testing.
+func searchCli(w io.Writer, n int, args []string) subcommands.ExitStatus {
+	index.load()
+	for _, q := range args {
+		items, more, _ := search(q, n)
+		if len(items) == 1 {
+			fmt.Fprintf(w, "Search for %s, page %d: 1 result\n", q, n)
+		} else {
+			fmt.Fprintf(w, "Search for %s, page %d: %d results\n", q, n, len(items))
+		}
+		for _, p := range items {
+			fmt.Fprintf(w, "* [%s](%s) (%d)\n", p.Title, p.Name, p.Score)
+		}
+		if more {
+			fmt.Fprintf(w, "There are more results\n")
+		}
+	}
+	return subcommands.ExitSuccess
+}

search_cmd_test.go

@@ -0,0 +1,19 @@
+package main
+
+import (
+	"bytes"
+	"github.com/stretchr/testify/assert"
+	"testing"
+	"github.com/google/subcommands"
+)
+
+func TestSearchCmd(t *testing.T) {
+	b := new(bytes.Buffer)
+	s := searchCli(b, 1, []string{"oddµ"})
+	assert.Equal(t, subcommands.ExitSuccess, s)
+	r := `Search for oddµ, page 1: 2 results
+* [Oddµ: A minimal wiki](README) (5)
+* [Welcome to Oddµ](index) (5)
+`
+	assert.Equal(t, r, b.String())
+}

search_test.go

@@ -2,8 +2,9 @@ package main
 
 import (
 	"github.com/stretchr/testify/assert"
-	"testing"
 	"net/url"
+	"testing"
+	"os"
 )
 
 func TestSearch(t *testing.T) {
@@ -12,3 +13,18 @@ func TestSearch(t *testing.T) {
 	assert.Contains(t,
 		assert.HTTPBody(searchHandler, "GET", "/search", data), "Welcome")
 }
+
+// wipes testdata
+func TestSearchQuestionmark(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	p := &Page{Name: "testdata/Odd?", Body: []byte(`# Even?
+
+yes or no?`)}
+	p.save()
+	data := url.Values{}
+	data.Set("q", "yes")
+	body := assert.HTTPBody(searchHandler, "GET", "/search", data)
+	assert.Contains(t, body, "yes or no?")
+	assert.NotContains(t, body, "Odd?")
+	assert.Contains(t, body, "Even?")
+}

snippets.go

@@ -75,7 +75,7 @@ func snippets(q string, s string) string {
 				to = len(s)
 			}
 			end := strings.LastIndex(s[:to], " ")
-			if end == -1 || end <= j + wl {
+			if end == -1 || end <= j+wl {
 				// OK, look for a longer word
 				end = strings.Index(s[to:], " ")
 				if end == -1 {

snippets_test.go

@@ -41,5 +41,5 @@ func TestSnippetsLong(t *testing.T) {
 	assert.Equal(t,
 		"VWwXetig mty8fORN UNia4NFm SQsfyFHk BLDdgVnc AcvKP2fs q8KxPH1A IaCzFj96 J0S2fqca jp3ElV9f ULIZ1aMX … Auk5RBWs tMpfMMlU p6jGYq3Z rTIBTHVM zGFwFwQi <b>j4O1AY21</b> BJnaiScY",
 		snippets("j4O1AY21", s))
-	
+
 }

upload.html

@@ -7,14 +7,32 @@
     <title>Upload File</title>
     <style>
 html { max-width: 70ch; padding: 2ch; margin: auto; color: #111; background: #ffe; }
+body { hyphens: auto; }
 form, textarea { width: 100%; }
+label { display: inline-block; width: 20ch }
     </style>
   </head>
-  <body>
+  <body lang="en">
     <h1>Upload File</h1>
     <form action="/drop/{{.}}" method="POST" enctype="multipart/form-data">
-      <input type="text" name="name" placeholder="image.jpg" autofocus required>
-      <p><input type="file" name="file" required>
+      <p>When uploading pictures from a phone, its filename is going to be something cryptic like IMG_1234.JPG.
+        Please provide your own filename.
+      <p><label for="text">Filename to use:</label>
+        <input id="text" name="name" type="text" placeholder="image.jpg" autofocus required>
+      <p>If the uploaded file is a picture from a phone, it is going to be too big for your site.
+        Sadly, resizing only works for JPG and PNG files. Luckily, most pictures from a phone camera are JPG images.
+        Feel free to specify a max width of 1200 pixels, for example.
+      <p><label for="maxwidth">Max width:</label>
+        <input id="maxwidth" name="maxwidth" type="number" min="10" placeholder="1200">
+      <p>If the uploaded file is a JPEG-encoded picture, like most pictures from a phone, you can specify a quality.
+        Typically, a quality of 60 is not too bad and a quality of 90 is more than enough.
+      <p><label for="quality">Quality:</label>
+        <input id="quality" name="quality" type="number" min="1" max="99" placeholder="75">
+      <p>Finally, pick the file or photo to upload.
+        Picture metadata is only removed if the picture gets resized.
+        Providing a new max width is recommended for all pictures.
+      <p><label for="file">Pick file to upload:</label>
+        <input type="file" name="file" required>
       <p><input type="submit" value="Save">
         <a href="/view/index"><button type="button">Cancel</button></a></p>
     </form>

upload_drop.go

@@ -1,10 +1,16 @@
 package main
 
 import (
+	"github.com/anthonynsimon/bild/imgio"
+	"github.com/anthonynsimon/bild/transform"
+	"image/jpeg"
 	"io"
 	"net/http"
 	"os"
 	"path"
+	"path/filepath"
+	"strconv"
+	"strings"
 )
 
 // uploadHandler uses the "upload.html" template to enable uploads.
@@ -28,7 +34,12 @@ func dropHandler(w http.ResponseWriter, r *http.Request, dir string) {
 		http.Error(w, "file exists", http.StatusInternalServerError)
 		return
 	}
-	filename := r.FormValue("name")
+	name := r.FormValue("name")
+	filename := filepath.Base(name)
+	if filename == "." || filepath.Dir(name) != "." {
+		http.Error(w, "no filename", http.StatusInternalServerError)
+		return
+	}
 	file, _, err := r.FormFile("file")
 	if err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -38,19 +49,64 @@ func dropHandler(w http.ResponseWriter, r *http.Request, dir string) {
 	// backup an existing file with the same name
 	_, err = os.Stat(filename)
 	if err != nil {
-		os.Rename(filename, filename + "~")
+		os.Rename(filename, filename+"~")
 	}
 	// create the new file
-	dst, err := os.Create(d + "/" + filename)
+	path := d + "/" + filename
+	dst, err := os.Create(path)
 	if err != nil {
-            http.Error(w, err.Error(), http.StatusInternalServerError)
-            return
-        }
+		http.Error(w, err.Error(), http.StatusInternalServerError)
+		return
+	}
 	defer dst.Close()
 	if _, err := io.Copy(dst, file); err != nil {
-            http.Error(w, err.Error(), http.StatusInternalServerError)
-            return
-        }
-	http.Redirect(w, r, "/view/"+filename, http.StatusFound)
-}
+		http.Error(w, err.Error(), http.StatusInternalServerError)
+		return
+	}
+	// if a resize was requested
+	maxwidth := r.FormValue("maxwidth")
+	if len(maxwidth) > 0 {
+		mw, err := strconv.Atoi(maxwidth)
+		if err != nil {
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+		ext := strings.ToLower(filepath.Ext(path))
+		var encoder imgio.Encoder
+		switch ext {
+		case ".png":
+			encoder = imgio.PNGEncoder()
+		case ".jpg", ".jpeg":
+			q := jpeg.DefaultQuality
+			quality := r.FormValue("quality")
+			if len(quality) > 0 {
+				q, err = strconv.Atoi(quality)
+				if err != nil {
+					http.Error(w, err.Error(), http.StatusInternalServerError)
+					return
+				}
+			}
+			encoder = imgio.JPEGEncoder(q)
+		default:
+			http.Error(w, "only .png, .jpg, or .jpeg files are supported", http.StatusInternalServerError)
+			return
+		}
+		img, err := imgio.Open(path)
+		if err != nil {
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+		rect := img.Bounds()
+		width := rect.Max.X - rect.Min.X
+		if width > mw {
+			height := (rect.Max.Y - rect.Min.Y) * mw / width
+			img = transform.Resize(img, mw, height, transform.Linear)
+			if err := imgio.Save(path, img, encoder); err != nil {
+				http.Error(w, err.Error(), http.StatusInternalServerError)
+				return
+			}
+		}
 
+	}
+	http.Redirect(w, r, "/view/"+path, http.StatusFound)
+}

upload_drop_test.go

@@ -3,6 +3,9 @@ package main
 import (
 	"bytes"
 	"github.com/stretchr/testify/assert"
+	"image"
+	"image/jpeg"
+	"image/png"
 	"mime/multipart"
 	"os"
 	"regexp"
@@ -14,19 +17,19 @@ func TestUpload(t *testing.T) {
 	_ = os.RemoveAll("testdata")
 	// for uploads, the directory is not created automatically
 	os.MkdirAll("testdata", 0755)
-	assert.HTTPStatusCode(t, makeHandler(uploadHandler, false), "GET", "/upload/", nil, 200)
+	assert.HTTPStatusCode(t, makeHandler(uploadHandler, false), "GET", "/upload/testdata/", nil, 200)
 	form := new(bytes.Buffer)
 	writer := multipart.NewWriter(form)
 	field, err := writer.CreateFormField("name")
 	assert.NoError(t, err)
-	_, err = field.Write([]byte("testdata/ok.txt"))
+	_, err = field.Write([]byte("ok.txt"))
 	assert.NoError(t, err)
-	file, err := writer.CreateFormFile("file", "example.txt");
+	file, err := writer.CreateFormFile("file", "example.txt")
 	assert.NoError(t, err)
 	file.Write([]byte("Hello!"))
 	err = writer.Close()
 	assert.NoError(t, err)
-	HTTPUploadAndRedirectTo(t, makeHandler(dropHandler, false), "/drop/",
+	HTTPUploadAndRedirectTo(t, makeHandler(dropHandler, false), "/drop/testdata/",
 		writer.FormDataContentType(), form, "/view/testdata/ok.txt")
 	assert.Regexp(t, regexp.MustCompile("Hello!"),
 		assert.HTTPBody(makeHandler(viewHandler, true), "GET", "/view/testdata/ok.txt", nil))
@@ -34,3 +37,43 @@ func TestUpload(t *testing.T) {
 		_ = os.RemoveAll("testdata")
 	})
 }
+
+// wipes testdata
+func TestUploadPng(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	// for uploads, the directory is not created automatically
+	os.MkdirAll("testdata", 0755)
+	form := new(bytes.Buffer)
+	writer := multipart.NewWriter(form)
+	field, _ := writer.CreateFormField("name")
+	field.Write([]byte("ok.png"))
+	file, _ := writer.CreateFormFile("file", "ok.png")
+	img := image.NewRGBA(image.Rect(0, 0, 20, 20))
+	png.Encode(file, img)
+	writer.Close()
+	HTTPUploadAndRedirectTo(t, makeHandler(dropHandler, false), "/drop/testdata/",
+		writer.FormDataContentType(), form, "/view/testdata/ok.png")
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}
+
+// wipes testdata
+func TestUploadJpg(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	// for uploads, the directory is not created automatically
+	os.MkdirAll("testdata", 0755)
+	form := new(bytes.Buffer)
+	writer := multipart.NewWriter(form)
+	field, _ := writer.CreateFormField("name")
+	field.Write([]byte("ok.jpg"))
+	file, _ := writer.CreateFormFile("file", "ok.jpg")
+	img := image.NewRGBA(image.Rect(0, 0, 20, 20))
+	jpeg.Encode(file, img, &jpeg.Options{Quality: 90})
+	writer.Close()
+	HTTPUploadAndRedirectTo(t, makeHandler(dropHandler, false), "/drop/testdata/",
+		writer.FormDataContentType(), form, "/view/testdata/ok.jpg")
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}

view.go

@@ -1,8 +1,10 @@
 package main
 
-import(
+import (
 	"net/http"
 	"os"
+	"time"
+	"strings"
 )
 
 // rootHandler just redirects to /view/index.
@@ -14,19 +16,67 @@ func rootHandler(w http.ResponseWriter, r *http.Request) {
 // the .md extension). If the requested file does not exist, a page
 // with the same name is loaded. This means adding the .md extension
 // and using the "view.html" template to render the HTML. Both
-// attempts fail, the browser is redirected to an edit page.
+// attempts fail, the browser is redirected to an edit page. As far as
+// caching goes: we respond with a 304 NOT MODIFIED if the request has
+// an If-Modified-Since header that matches the file's modification
+// time, truncated to one second, because the file's modtime has
+// sub-second precision and the HTTP timestamp for the Last-Modified
+// header has not.
 func viewHandler(w http.ResponseWriter, r *http.Request, name string) {
-	body, err := os.ReadFile(name)
+	file := true
+	rss := false
+	fn := name
+	fi, err := os.Stat(fn)
+	if err != nil {
+		file = false
+		if strings.HasSuffix(fn, ".rss") {
+			rss = true
+			name = fn[0:len(fn)-4]
+			fn = name
+		}
+		fn += ".md"
+		fi, err = os.Stat(fn)
+	}
 	if err == nil {
+		h, ok := r.Header["If-Modified-Since"]
+		if ok {
+			ti, err := http.ParseTime(h[0])
+			if err == nil && !fi.ModTime().Truncate(time.Second).After(ti) {
+				w.WriteHeader(http.StatusNotModified)
+				return
+			}
+		}
+		w.Header().Set("Last-Modified", fi.ModTime().UTC().Format(http.TimeFormat))
+	}
+	if r.Method == http.MethodHead {
+		if err == nil {
+			return
+		}
+		http.Redirect(w, r, "/edit/"+name, http.StatusFound)
+	}
+	if file {
+		body, err := os.ReadFile(fn)
+		if err != nil {
+			// This is an internal error because os.Stat
+			// says there is a file. Non-existent files
+			// are treated like pages.
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+		}
 		w.Write(body)
 		return
 	}
 	p, err := loadPage(name)
-	if err == nil {
-		p.handleTitle(true)
-		p.renderHtml()
-		renderTemplate(w, "view", p)
+	if err != nil {
+		http.Redirect(w, r, "/edit/"+name, http.StatusFound)
 		return
 	}
-	http.Redirect(w, r, "/edit/"+name, http.StatusFound)
+	p.handleTitle(true)
+	if rss {
+		it := feed(p, fi.ModTime())
+		w.Write([]byte(`<?xml version="1.0" encoding="UTF-8"?>`))
+		renderTemplate(w, "feed", it)
+		return
+	}
+	p.renderHtml()
+	renderTemplate(w, "view", p)
 }

view.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html>
+<html lang="{{.Language}}">
   <head>
     <meta charset="utf-8">
     <meta name="format-detection" content="telephone=no">
@@ -10,20 +10,23 @@ html { max-width: 65ch; padding: 1ch; margin: auto; color: #111; background: #ff
 body { hyphens: auto; }
 header a { margin-right: 1ch; }
 form { display: inline-block; }
+input#search { width: 12ch; }
+button { background: #eee; color: inherit; border-radius: 4px; border-width: 1px; }
 footer { border-top: 1px solid #888 }
 img { max-width: 100%; }
     </style>
   </head>
-  <body lang="{{.Language}}">
+  <body>
     <header>
       <a href="#main">Skip navigation</a>
       <a href="/view/index">Home</a>
-      <a href="/edit/{{.Name}}">Edit</a>
-      <a href="/add/{{.Name}}">Add</a>
-      <a href="/upload/{{.Dir}}">Upload</a>
+      <a href="/edit/{{.Name}}" accesskey="e">Edit</a>
+      <a href="/add/{{.Name}}" accesskey="a">Add</a>
+      <a href="/upload/{{.Dir}}" accesskey="u">Upload</a>
       <form role="search" action="/search" method="GET">
-        <input type="text" spellcheck="false" name="q" required>
-        <button>Search</button>
+        <label for="search">Search:</label>
+        <input id="search" type="text" spellcheck="false" name="q" accesskey="f" required>
+        <button>Go</button>
       </form>
     </header>
     <main id="main">
@@ -32,7 +35,7 @@ img { max-width: 100%; }
     </main>
     <footer>
       <address>
-        Comments? Send mail to Alex Schroeder <<a href="mailto:alex@alexschroeder.ch">alex@alexschroeder.ch</a>>
+        Comments? Send mail to Your Name <<a href="mailto:you@example.org">you@example.org</a>>
       </address>
     </footer>
   </body>

view_test.go

@@ -2,6 +2,7 @@ package main
 
 import (
 	"github.com/stretchr/testify/assert"
+	"net/http"
 	"os"
 	"regexp"
 	"testing"
@@ -38,6 +39,7 @@ func TestPageTitleWithAmp(t *testing.T) {
 	})
 }
 
+// wipes testdata
 func TestPageTitleWithQuestionMark(t *testing.T) {
 	_ = os.RemoveAll("testdata")
 
@@ -46,9 +48,75 @@ func TestPageTitleWithQuestionMark(t *testing.T) {
 
 	body := assert.HTTPBody(makeHandler(viewHandler, true), "GET", "/view/testdata/How%20about%20no%3F", nil)
 	assert.Contains(t, body, "No means no")
-	assert.Contains(t, body, "<a href=\"/edit/testdata/How%20about%20no%3F\">Edit</a>")
+	assert.Contains(t, body, "<a href=\"/edit/testdata/How%20about%20no%3F\" accesskey=\"e\">Edit</a>")
 
 	t.Cleanup(func() {
 		_ = os.RemoveAll("testdata")
 	})
 }
+
+// wipes testdata
+func TestFileLastModified(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	assert.NoError(t, os.Mkdir("testdata", 0755))
+	assert.NoError(t, os.WriteFile("testdata/now.txt", []byte(`
+A spider sitting
+Unmoving and still
+In the autumn chill
+`), 0644))
+	fi, err := os.Stat("testdata/now.txt")
+	assert.NoError(t, err)
+	h := makeHandler(viewHandler, true)
+	assert.Equal(t, []string{fi.ModTime().UTC().Format(http.TimeFormat)},
+		HTTPHeaders(h, "GET", "/view/testdata/now.txt", nil, "Last-Modified"))
+	HTTPStatusCodeIfModifiedSince(t, h, "/view/testdata/now.txt", fi.ModTime())
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}
+
+// wipes testdata
+func TestPageLastModified(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	p := &Page{Name: "testdata/now", Body: []byte(`
+The sky glows softly
+Sadly, the birds are quiet
+I like spring better
+`)}
+	p.save()
+	fi, err := os.Stat("testdata/now.md")
+	assert.NoError(t, err)
+	h := makeHandler(viewHandler, true)
+	assert.Equal(t, []string{fi.ModTime().UTC().Format(http.TimeFormat)},
+	HTTPHeaders(h, "GET", "/view/testdata/now", nil, "Last-Modified"))
+	HTTPStatusCodeIfModifiedSince(t, h, "/view/testdata/now", fi.ModTime())
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}
+
+
+// wipes testdata
+func TestPageHead(t *testing.T) {
+	_ = os.RemoveAll("testdata")
+	p := &Page{Name: "testdata/peace", Body: []byte(`
+No urgent typing
+No todos, no list, no queue.
+Just me and the birds.
+`)}
+	p.save()
+	fi, err := os.Stat("testdata/peace.md")
+	assert.NoError(t, err)
+	h := makeHandler(viewHandler, true)
+	assert.Equal(t, []string(nil),
+		HTTPHeaders(h, "HEAD", "/view/testdata/war", nil, "Last-Modified"))
+	assert.Equal(t, []string(nil),
+		HTTPHeaders(h, "GET", "/view/testdata/war", nil, "Last-Modified"))
+	assert.Equal(t, []string{fi.ModTime().UTC().Format(http.TimeFormat)},
+		HTTPHeaders(h, "HEAD", "/view/testdata/peace", nil, "Last-Modified"))
+	assert.Equal(t, "",
+		assert.HTTPBody(h, "HEAD", "/view/testdata/peace", nil))
+	t.Cleanup(func() {
+		_ = os.RemoveAll("testdata")
+	})
+}

wiki.go

@@ -1,7 +1,10 @@
 package main
 
 import (
+	"context"
+	"flag"
 	"fmt"
+	"github.com/google/subcommands"
 	"html/template"
 	"net/http"
 	"os"
@@ -11,7 +14,7 @@ import (
 // Templates are parsed at startup.
 var templates = template.Must(
 	template.ParseFiles("edit.html", "add.html", "view.html",
-		"search.html", "upload.html"))
+		"search.html", "upload.html", "feed.html"))
 
 // validPath is a regular expression where the second group matches a
 // page, so when the editHandler is called, a URL path of "/edit/foo"
@@ -96,11 +99,29 @@ func serve() {
 	http.HandleFunc("/search", searchHandler)
 	go scheduleLoadIndex()
 	go scheduleLoadLanguages()
+	initAccounts()
 	port := getPort()
 	fmt.Printf("Serving a wiki on port %s\n", port)
 	http.ListenAndServe(":"+port, nil)
 }
 
+// commands does the command line parsing in case Oddmu is called with
+// some arguments. Without any arguments, the wiki server is started.
+// At this point we already know that there is at least one
+// subcommand.
+func commands() {
+	subcommands.Register(subcommands.HelpCommand(), "")
+	subcommands.Register(subcommands.FlagsCommand(), "")
+	subcommands.Register(subcommands.CommandsCommand(), "")
+	subcommands.Register(&htmlCmd{}, "")
+	subcommands.Register(&searchCmd{}, "")
+	subcommands.Register(&replaceCmd{}, "")
+
+	flag.Parse()
+	ctx := context.Background()
+	os.Exit(int(subcommands.Execute(ctx)))
+}
+
 func main() {
 	if len(os.Args) == 1 {
 		serve()

wiki_test.go

@@ -8,6 +8,7 @@ import (
 	"net/url"
 	"strings"
 	"testing"
+	"time"
 )
 
 // HTTPHeaders is a helper that returns HTTP headers of the response. It returns
@@ -65,3 +66,13 @@ func HTTPUploadAndRedirectTo(t *testing.T, handler http.HandlerFunc, url, conten
 		"Expected HTTP redirect location %s for %q but received %v", destination, url, headers)
 	return isRedirectCode
 }
+// HTTPStatusCodeIfModifiedSince checks that the request results in a
+// 304 response for the given time.
+func HTTPStatusCodeIfModifiedSince(t *testing.T, handler http.HandlerFunc, url string, ti time.Time) {
+	w := httptest.NewRecorder()
+	req, err := http.NewRequest("GET", url, nil)
+	assert.NoError(t, err)
+	req.Header.Set("If-Modified-Since", ti.UTC().Format(http.TimeFormat))
+	handler(w, req)
+	assert.Equal(t, http.StatusNotModified, w.Code)
+}