Add oddmu-releases(7)

Since prefixes always have to match, the boolean expression can be
simplified and tests still pass.

README.md

@@ -59,6 +59,10 @@ indexes – lists of page links. These are important for feeds.
 [oddmu-search(7)](/oddmu.git/blob/main/man/oddmu-search.7.txt): This
 man page documents how search and scoring work.
 
+[oddmu-filter(7)](/oddmu.git/blob/main/man/oddmu-filter.7.txt): This
+man page documents how to exclude subdirectories from search and
+archiving.
+
 [oddmu-replace(1)](/oddmu.git/blob/main/man/oddmu-replace.1.txt): This
 man page documents the "replace" subcommand to make mass changes to
 the files much like find(1), grep(1) and sed(1) or perl(1).
@@ -122,34 +126,35 @@ If you spot any, [contact](https://alexschroeder.ch/wiki/Contact) me.
 If you're interested in making changes to the code, here's a
 high-level introduction to the various source files.
 
-- *_test.go are the test files; a few library functions are defined in
-  wiki_test.go.
-- *_cmd.go are the files implementing the various subcommands with
+- `*_test.go` are the test files; a few library functions are defined
+  in `wiki_test.go`.
+- `*_cmd.go` are the files implementing the various subcommands with
   matching names
-- accounts.go implements the webfinger code to fetch fediverse account
-  link destinations with the URI provided by webfinger
-- add_append.go implements the /add and /append handlers
-- diff.go implements the /diff handler
-- edit_save.go implements the /edit and /save handlers
-- feed.go implements the feed for a page based on the links it lists
-- highlight.go implements the bold tags for matches when showing
+- `accounts.go` implements the webfinger code to fetch fediverse
+  account link destinations with the URI provided by webfinger
+- `add_append.go` implements the `/add` and `/append` handlers
+- `archive.go` implements the `/archive` handler
+- `diff.go` implements the `/diff` handler
+- `edit_save.go` implements the `/edit` and `/save` handlers
+- `feed.go` implements the feed for a page based on the links it lists
+- `highlight.go` implements the bold tags for matches when showing
   search results
-- index.go implements the index of all the hashtags
-- languages.go implements the language detection
-- page.go implements the page loading and saving
-- parser.go implements the Markdown parsing
-- score.go implements the page scoring when showing search results
-- search.go implements the /search handler
-- snippets.go implements the page summaries for search results
-- templates.go implements template loading and reloading
-- tokenizer.go implements the various tokenizers used
-- upload_drop.go implements the /upload and /drop handlers
-- view.go implements the /view handler
-- watch.go implements the filesystem notification watch
-- wiki.go implements the main function
+- `index.go` implements the index of all the hashtags
+- `languages.go` implements the language detection
+- `page.go` implements the page loading and saving
+- `parser.go` implements the Markdown parsing
+- `score.go` implements the page scoring when showing search results
+- `search.go` implements the `/search` handler
+- `snippets.go` implements the page summaries for search results
+- `templates.go` implements template loading and reloading
+- `tokenizer.go` implements the various tokenizers used
+- `upload_drop.go` implements the `/upload` and `/drop` handlers
+- `view.go` implements the `/view` handler
+- `watch.go` implements the filesystem notification watch
+- `wiki.go` implements the main function
 
-If you want to change the exact markup rules, your starting point
-should be `parser.go`. Make sure you read the documentation of [Go
+If you want to change the markup rules, your starting point should be
+`parser.go`. Make sure you read the documentation of [Go
 Markdown](https://github.com/gomarkdown/markdown) and note that it
 offers MathJax support (needs a change to the `view.html` template so
 that the MathJax Javascript gets loaded) and
@@ -160,14 +165,18 @@ One of the sad parts of the code is the distinction between path and
 filepath. On a Linux system, this doesn't matter. I suspect that it
 also doesn't matter on MacOS and Windows because the file systems
 handle forward slashes just fine. The code still tries to do the right
-thing. A path that is derived from a URL is a path, with slashes.
-Before accessing a file, it has to turned into a filepath using
-filepath.FromSlashes and in the rare case where the inverse happens,
-use filepath.ToSlashes.
+thing. A path that is derived from a URL is a path with slashes.
+Before accessing a file, it has to be turned into a filepath using
+`filepath.FromSlashes` and in the rare case where the inverse happens,
+use `filepath.ToSlashes`. Any path received via the URL path uses
+slashes and needs to be converted to a filepath before passing it to
+any `os` function. Any path received within a `path/filepath.WalkFunc`
+is a filepath and needs to be converted to use slashes when used in
+HTML output.
 
-In the rare cases where you need to access the page name in code that
-is used from a template, you have to decode the path. See the code in
-diff.go for an example.
+If you need to access the page name in code that is used from a
+template, you have to decode the path. See the code in `diff.go` for
+an example.
 
 ## References
 

RELEASE

@@ -0,0 +1,12 @@
+When preparing a new release
+----------------------------
+
+1. Run tests
+
+2. Make docs
+
+3. Make sure all files are checked in
+
+4. Update man/oddmu-releases.7.txt
+
+5. Tag the release and push the tag to all remotes

archive.go

@@ -0,0 +1,69 @@
+package main
+
+import (
+	"archive/zip"
+	"io"
+	"io/fs"
+	"log"
+	"net/http"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+)
+
+// archiveHandler serves a zip file. Directories starting with a period are skipped. Filenames starting with a period
+// are skipped. If the environment variable ODDMU_FILTER is a regular expression that matches the starting directory,
+// this is a "separate site"; if the regular expression does not match, this is the "main site" and page names must also
+// not match the regular expression.
+func archiveHandler(w http.ResponseWriter, r *http.Request, path string) {
+	filter := os.Getenv("ODDMU_FILTER")
+	re, err := regexp.Compile(filter)
+	if err != nil {
+		log.Println("ODDMU_FILTER does not compile:", filter, err)
+		http.Error(w, err.Error(), http.StatusInternalServerError)
+		return
+	}
+	matches := re.MatchString(path)
+	dir := filepath.Dir(filepath.FromSlash(path))
+	z := zip.NewWriter(w)
+	err = filepath.Walk(dir, func (path string, info fs.FileInfo, err error) error {
+			if err != nil {
+				return err
+			}
+			if info.IsDir() {
+				if path != "." && strings.HasPrefix(filepath.Base(path), ".") {
+					return filepath.SkipDir
+				}
+			} else if !strings.HasPrefix(filepath.Base(path), ".") &&
+				(matches || !re.MatchString(path)) {
+				zf, err := z.Create(path)
+				if err != nil {
+					log.Println(err)
+					return err
+				}
+				file, err := os.Open(path)
+				if err != nil {
+					log.Println(err)
+					return err
+				}
+				_, err = io.Copy(zf, file)
+				if err != nil {
+					log.Println(err)
+					return err
+				}
+			}
+			return nil
+		})
+	if err != nil {
+		log.Println(err)
+		http.Error(w, err.Error(), http.StatusInternalServerError)
+		return
+	}
+	err = z.Close()
+	if err != nil {
+		log.Println(err)
+		http.Error(w, err.Error(), http.StatusInternalServerError)
+		return
+	}
+}

archive_test.go

@@ -0,0 +1,27 @@
+package main
+
+import (
+	"archive/zip"
+	"github.com/stretchr/testify/assert"
+	"os"
+	"testing"
+	"strings"
+)
+
+func TestArchive(t *testing.T) {
+	cleanup(t, "testdata/archive")
+	assert.NoError(t, os.MkdirAll("testdata/archive/public", 0755))
+	assert.NoError(t, os.MkdirAll("testdata/archive/secret", 0755))
+	assert.NoError(t, os.WriteFile("testdata/archive/public/index.md", []byte("# Public\nChurch tower bells ringing\nA cold wind biting my ears\nWalk across the square"), 0644))
+	assert.NoError(t, os.WriteFile("testdata/archive/secret/index.md", []byte("# Secret\nMany years ago I danced\nSpending nights in clubs and bars\nIt is my secret"), 0644))
+	os.Setenv("ODDMU_FILTER", "^testdata/archive/secret/")
+	body := assert.HTTPBody(makeHandler(archiveHandler, true), "GET", "/archive/testdata/data.zip", nil)
+	r, err := zip.NewReader(strings.NewReader(body), int64(len(body)))
+	assert.NoError(t, err, "Unzip")
+	names := []string{}
+	for _, file := range r.File {
+		names = append(names, file.Name)
+	}
+	assert.Contains(t, names, "testdata/archive/public/index.md")
+	assert.NotContains(t, names, "testdata/archive/secret/index.md")
+}

man/Makefile

@@ -19,13 +19,17 @@ md: ${MD}
 	sed --regexp-extended \
 	  -e 's/\*([^*]+)\*/**\1**/g' \
 	  -e 's/_(oddmu[a-z.-]*)_\(([1-9])\)/[\1(\2)](\1.\2)/g' \
-	  -e 's/_([^_]+)_/*\1*/g' \
+	  -e 's/\b_([^_]+)_\b/*\1*/g' \
 	  -e 's/^#/##/' \
 	  -e 's/^([A-Z.-]*\([1-9]\))( ".*")?$$/# \1/' \
 	  < $< > $@
 
 upload: ${MD}
 	rsync --itemize-changes --archive *.md sibirocobombus:alexschroeder.ch/wiki/oddmu/
+	make clean
 
 clean:
 	rm --force ${HTML} ${MD}
+
+realclean: clean
+	rm --force ${MAN}

man/oddmu-apache.5

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-APACHE" "5" "2024-02-13"
+.TH "ODDMU-APACHE" "5" "2024-02-17"
 .PP
 .SH NAME
 .PP
@@ -48,7 +48,7 @@ ServerAdmin alex@alexschroeder\&.ch
 <VirtualHost *:443>
   ServerName transjovian\&.org
   SSLEngine on
-  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(\&.*))?$" 
+  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(\&.*))?$" 
                  "http://localhost:8080/$1"
 </VirtualHost>
 .fi
@@ -81,6 +81,35 @@ apachectl graceful
 .fi
 .RE
 .PP
+In a situation where Apache acts as a reverse proxy, you can prevent some
+actions from being proxied.\& If you don'\&t want to allow strangers to make
+changes, search or archive the site, use a limited setup like the following:
+.PP
+.nf
+.RS 4
+MDomain transjovian\&.org
+MDCertificateAgreement accepted
+ServerAdmin alex@alexschroeder\&.ch
+
+<VirtualHost *:80>
+  ServerName transjovian\&.org
+  Redirect "/" "https://transjovian\&.org/"
+</VirtualHost>
+<VirtualHost *:443>
+  ServerName transjovian\&.org
+  SSLEngine on
+  ProxyPassMatch "^/(view/\&.*)?$" "http://localhost:8080/$1"
+</VirtualHost>
+.fi
+.RE
+.PP
+You'\&ll need to edit the source pages some other way.\& Edit them locally and
+upload them using rsync; edit them remotely using an editor that can do this;
+use SSHFS to mount the remote directory locally for editing; use \fIstunnel\fR(8) to
+access the remote wiki on the local port 8080 for editing.\& There are probably a
+lot more such options available.\& All of them have the drawback that they'\&re
+probably not easy to use when on a mobile phone.\&
+.PP
 .SS Allow HTTP for viewing
 .PP
 When looking at pages, you might want to allow HTTP since no password is
@@ -95,7 +124,7 @@ ServerAdmin alex@alexschroeder\&.ch
 
 <VirtualHost *:80>
   ServerName transjovian\&.org
-  ProxyPassMatch "^/((view|diff|search)/(\&.*))?$" 
+  ProxyPassMatch "^/((view|diff|search|archive)/(\&.*))?$" 
                  "http://localhost:8080/$1"
   RedirectMatch  "^/((edit|save|add|append|upload|drop)/(\&.*))?$" 
                  "https://transjovian\&.org/$1"
@@ -103,7 +132,7 @@ ServerAdmin alex@alexschroeder\&.ch
 <VirtualHost *:443>
   ServerName transjovian\&.org
   SSLEngine on
-  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(\&.*))?$" 
+  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(\&.*))?$" 
                  "http://localhost:8080/$1"
 </VirtualHost>
 .fi
@@ -150,7 +179,7 @@ In that case, you need to use the ProxyPassMatch directive.\&
 .PP
 .nf
 .RS 4
-ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(\&.*))?$" 
+ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(\&.*))?$" 
                "unix:/run/oddmu/oddmu\&.sock|http://localhost/$1"
 .fi
 .RE
@@ -169,7 +198,7 @@ A workaround is to add the redirect manually and drop the question-mark:
 .nf
 .RS 4
 RedirectMatch "^/$" "/view/index"
-ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(\&.*))$" 
+ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(\&.*))$" 
                "unix:/run/oddmu/oddmu\&.sock|http://localhost/$1"
 .fi
 .RE
@@ -235,26 +264,26 @@ from public view.\& Search reveals the existence of these pages and produces an
 extract, even if users cannot follow the links.\& Archive links pack all the
 subdirectories, including locations you may have hidden from view using Apache.\&
 .PP
-If you want private subdirectories, you need to set the environment variable
-ODDMU_FILTER to a regular expression matching the private directories.\& If search
-starts in a directory matching the regular expression, the directory and its
-subdirectories are searched.\& If search starts in a directory that doesn'\&t match
-the regular expression, all directories matching the regular expression are
-excluded.\&
+If you to treat subdirectories as separate sites, you need to set the
+environment variable ODDMU_FILTER to a regular expression matching the those
+directories.\& If search starts in a directory that doesn'\&t match the regular
+expression, all directories matching the regular expression are excluded.\& See
+\fIoddmu-filter\fR(7).\&
 .PP
 In the following example, ODDMU_FILTER is set to "^secret/".\&
 .PP
-http://transjovian.\&org/search/index?\&q=something does not search the "secret"
+http://transjovian.\&org/search/index?\&q=something does not search the "secret/"
 directory and its subdirectories are excluded.\&
 .PP
 http://transjovian.\&org/search/secret/index?\&q=something searches just the
 "secret" directory and its subdirectories.\&
 .PP
-Now you need to ensure that the "secret" directory are not public.\&
+You need to configure the web server to prevent access to the "secret/"
+directory:
 .PP
 .nf
 .RS 4
-<LocationMatch "^/(edit|save|add|append|upload|drop|view/secret|search/secret)/">
+<LocationMatch "^/(edit|save|add|append|upload|drop|(view|search|archive)/secret)/">
   AuthType Basic
   AuthName "Password Required"
   AuthUserFile /home/oddmu/\&.htpasswd
@@ -279,9 +308,9 @@ DocumentRoot /home/oddmu
 .RE
 .PP
 Make sure that none of the subdirectories look like the wiki paths "/view/",
-"/diff/", "/edit/", "/save/", "/add/", "/append/", "/upload/", "/drop/" or
-"/search/".\& For example, create a file called "robots.\&txt" containing the
-following, telling all robots that they'\&re not welcome.\&
+"/diff/", "/edit/", "/save/", "/add/", "/append/", "/upload/", "/drop/",
+"/search/" or "/archive/".\& For example, create a file called "robots.\&txt"
+containing the following, telling all robots that they'\&re not welcome.\&
 .PP
 .nf
 .RS 4
@@ -337,8 +366,8 @@ This requires a valid login by the user "alex" or "berta":
 .PP
 .SS Private wikis
 .PP
-Based on the above, you can prevent people from \fIreading\fR the wiki.\& The
-"LocationMatch" must cover all the URLs in order to protect everything.\&
+Based on the above, you can prevent people from \fIreading\fR the wiki.\& The location
+must cover all the URLs in order to protect everything.\&
 .PP
 .nf
 .RS 4
@@ -361,7 +390,7 @@ such that ever domain acts as a reverse proxy to a different Oddmu instance.\&
 .PP
 .SH SEE ALSO
 .PP
-\fIoddmu\fR(1)
+\fIoddmu\fR(1), \fIoddmu-filter\fR(7)
 .PP
 "Apache Core Features".\&
 https://httpd.\&apache.\&org/docs/current/mod/core.\&html

man/oddmu-apache.5.txt

@@ -40,7 +40,7 @@ ServerAdmin alex@alexschroeder.ch
 <VirtualHost *:443>
   ServerName transjovian.org
   SSLEngine on
-  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(.*))?$" \
+  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
                  "http://localhost:8080/$1"
 </VirtualHost>
 ```
@@ -64,6 +64,33 @@ Restart the server, gracefully:
 apachectl graceful
 ```
 
+In a situation where Apache acts as a reverse proxy, you can prevent some
+actions from being proxied. If you don't want to allow strangers to make
+changes, search or archive the site, use a limited setup like the following:
+
+```
+MDomain transjovian.org
+MDCertificateAgreement accepted
+ServerAdmin alex@alexschroeder.ch
+
+<VirtualHost *:80>
+  ServerName transjovian.org
+  Redirect "/" "https://transjovian.org/"
+</VirtualHost>
+<VirtualHost *:443>
+  ServerName transjovian.org
+  SSLEngine on
+  ProxyPassMatch "^/(view/.*)?$" "http://localhost:8080/$1"
+</VirtualHost>
+```
+
+You'll need to edit the source pages some other way. Edit them locally and
+upload them using rsync; edit them remotely using an editor that can do this;
+use SSHFS to mount the remote directory locally for editing; use _stunnel_(8) to
+access the remote wiki on the local port 8080 for editing. There are probably a
+lot more such options available. All of them have the drawback that they're
+probably not easy to use when on a mobile phone.
+
 ## Allow HTTP for viewing
 
 When looking at pages, you might want to allow HTTP since no password is
@@ -77,7 +104,7 @@ ServerAdmin alex@alexschroeder.ch
 
 <VirtualHost *:80>
   ServerName transjovian.org
-  ProxyPassMatch "^/((view|diff|search)/(.*))?$" \
+  ProxyPassMatch "^/((view|diff|search|archive)/(.*))?$" \
                  "http://localhost:8080/$1"
   RedirectMatch  "^/((edit|save|add|append|upload|drop)/(.*))?$" \
                  "https://transjovian.org/$1"
@@ -85,7 +112,7 @@ ServerAdmin alex@alexschroeder.ch
 <VirtualHost *:443>
   ServerName transjovian.org
   SSLEngine on
-  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(.*))?$" \
+  ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
                  "http://localhost:8080/$1"
 </VirtualHost>
 ```
@@ -124,7 +151,7 @@ You probably want to serve some static files as well (see *Serve static files*).
 In that case, you need to use the ProxyPassMatch directive.
 
 ```
-ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(.*))?$" \
+ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
                "unix:/run/oddmu/oddmu.sock|http://localhost/$1"
 ```
 
@@ -139,7 +166,7 @@ A workaround is to add the redirect manually and drop the question-mark:
 
 ```
 RedirectMatch "^/$" "/view/index"
-ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search)/(.*))$" \
+ProxyPassMatch "^/((view|diff|edit|save|add|append|upload|drop|search|archive)/(.*))$" \
                "unix:/run/oddmu/oddmu.sock|http://localhost/$1"
 ```
 
@@ -196,25 +223,25 @@ from public view. Search reveals the existence of these pages and produces an
 extract, even if users cannot follow the links. Archive links pack all the
 subdirectories, including locations you may have hidden from view using Apache.
 
-If you want private subdirectories, you need to set the environment variable
-ODDMU_FILTER to a regular expression matching the private directories. If search
-starts in a directory matching the regular expression, the directory and its
-subdirectories are searched. If search starts in a directory that doesn't match
-the regular expression, all directories matching the regular expression are
-excluded.
+If you to treat subdirectories as separate sites, you need to set the
+environment variable ODDMU_FILTER to a regular expression matching the those
+directories. If search starts in a directory that doesn't match the regular
+expression, all directories matching the regular expression are excluded. See
+_oddmu-filter_(7).
 
 In the following example, ODDMU_FILTER is set to "^secret/".
 
-http://transjovian.org/search/index?q=something does not search the "secret"
+http://transjovian.org/search/index?q=something does not search the "secret/"
 directory and its subdirectories are excluded.
 
 http://transjovian.org/search/secret/index?q=something searches just the
 "secret" directory and its subdirectories.
 
-Now you need to ensure that the "secret" directory are not public.
+You need to configure the web server to prevent access to the "secret/"
+directory:
 
 ```
-<LocationMatch "^/(edit|save|add|append|upload|drop|view/secret|search/secret)/">
+<LocationMatch "^/(edit|save|add|append|upload|drop|(view|search|archive)/secret)/">
   AuthType Basic
   AuthName "Password Required"
   AuthUserFile /home/oddmu/.htpasswd
@@ -236,9 +263,9 @@ DocumentRoot /home/oddmu
 ```
 
 Make sure that none of the subdirectories look like the wiki paths "/view/",
-"/diff/", "/edit/", "/save/", "/add/", "/append/", "/upload/", "/drop/" or
-"/search/". For example, create a file called "robots.txt" containing the
-following, telling all robots that they're not welcome.
+"/diff/", "/edit/", "/save/", "/add/", "/append/", "/upload/", "/drop/",
+"/search/" or "/archive/". For example, create a file called "robots.txt"
+containing the following, telling all robots that they're not welcome.
 
 ```
 User-agent: *
@@ -288,8 +315,8 @@ This requires a valid login by the user "alex" or "berta":
 
 ## Private wikis
 
-Based on the above, you can prevent people from _reading_ the wiki. The
-"LocationMatch" must cover all the URLs in order to protect everything.
+Based on the above, you can prevent people from _reading_ the wiki. The location
+must cover all the URLs in order to protect everything.
 
 ```
 <Location />
@@ -310,7 +337,7 @@ such that ever domain acts as a reverse proxy to a different Oddmu instance.
 
 # SEE ALSO
 
-_oddmu_(1)
+_oddmu_(1), _oddmu-filter_(7)
 
 "Apache Core Features".
 https://httpd.apache.org/docs/current/mod/core.html

man/oddmu-filter.7

@@ -0,0 +1,63 @@
+.\" Generated by scdoc 1.11.3
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-FILTER" "7" "2024-02-17"
+.PP
+.SH NAME
+.PP
+oddmu-filter - keeping subdirectories separate
+.PP
+.SH DESCRIPTION
+.PP
+There are actions such as searching and archiving that act on multiple pages,
+not just a single page.\& These actions walk the directory tree, including all
+subdirectories.\& In some cases, this is not desirable.\&
+.PP
+Sometimes, subdirectories are separate sites, like the sites of other projects
+or different people.\& Essentially, the subdirectory acts as a different site.\&
+Depending on how you think about it, you might not want to include those "sites"
+in searches or archives of the whole site.\&
+.PP
+What'\&s important in this situation is whether the visitor is looking at the
+"main site" (a page further up in the directory tree) or at a particular page in
+a "separate site".\&
+.PP
+Since directory tree actions always start in the directory the visitor is
+currenly looking at, directory tree actions starting in a "separate site"
+automatically act as expected.\& The action is limited to that subdirectory tree.\&
+.PP
+When visitors look at a page in the "main site", however, directory tree actions
+must skip any sub directories that are part of a "separate site".\&
+.PP
+The way to identify separate sates is via the environment variable ODDMU_FILTER.\&
+It'\&s value is a regular expression matching separate sites.\&
+.PP
+.SH EXAMPLES
+.PP
+"ODDMU_FILTER=^project/" means that a directory tree action outside the
+"project/" directory does not include pages in the "project/" directory.\&
+.PP
+In other words, http://localhost:8080/search/?\&q=oddmu skips any pages in
+"project/".\&
+.PP
+At the same time, http://localhost:8080/search/project/?\&q=oddmu works like it
+always does: search is limited to "project/" and its subdirectories.\&
+.PP
+.SH SECURITY
+.PP
+If the subdirectory is a private site, then you need to use ODDMU_FILTER to
+exclude it from directory tree actions in the main site, and you need to
+configure your web server such that it doesn'\&t allow visitors access to the
+directory tree without authentication.\& See \fIoddmu-apache\fR(5).\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1), \fIoddmu-search\fR(7), \fIoddmu-apache\fR(5)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-filter.7.txt

@@ -0,0 +1,56 @@
+ODDMU-FILTER(7)
+
+# NAME
+
+oddmu-filter - keeping subdirectories separate
+
+# DESCRIPTION
+
+There are actions such as searching and archiving that act on multiple pages,
+not just a single page. These actions walk the directory tree, including all
+subdirectories. In some cases, this is not desirable.
+
+Sometimes, subdirectories are separate sites, like the sites of other projects
+or different people. Essentially, the subdirectory acts as a different site.
+Depending on how you think about it, you might not want to include those "sites"
+in searches or archives of the whole site.
+
+What's important in this situation is whether the visitor is looking at the
+"main site" (a page further up in the directory tree) or at a particular page in
+a "separate site".
+
+Since directory tree actions always start in the directory the visitor is
+currenly looking at, directory tree actions starting in a "separate site"
+automatically act as expected. The action is limited to that subdirectory tree.
+
+When visitors look at a page in the "main site", however, directory tree actions
+must skip any sub directories that are part of a "separate site".
+
+The way to identify separate sates is via the environment variable ODDMU_FILTER.
+It's value is a regular expression matching separate sites.
+
+# EXAMPLES
+
+"ODDMU_FILTER=^project/" means that a directory tree action outside the
+"project/" directory does not include pages in the "project/" directory.
+
+In other words, http://localhost:8080/search/?q=oddmu skips any pages in
+"project/".
+
+At the same time, http://localhost:8080/search/project/?q=oddmu works like it
+always does: search is limited to "project/" and its subdirectories.
+
+# SECURITY
+
+If the subdirectory is a private site, then you need to use ODDMU_FILTER to
+exclude it from directory tree actions in the main site, and you need to
+configure your web server such that it doesn't allow visitors access to the
+directory tree without authentication. See _oddmu-apache_(5).
+
+# SEE ALSO
+
+_oddmu_(1), _oddmu-search_(7), _oddmu-apache_(5)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-html.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-HTML" "1" "2023-10-09"
+.TH "ODDMU-HTML" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-list.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-LIST" "1" "2023-12-20"
+.TH "ODDMU-LIST" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-missing.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-MISSING" "1" "2023-11-05"
+.TH "ODDMU-MISSING" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-notify.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-NOTIFY" "1" "2024-01-17"
+.TH "ODDMU-NOTIFY" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-releases.7

@@ -0,0 +1,160 @@
+.\" Generated by scdoc 1.11.3
+.\" Complete documentation for this program is not available as a GNU info page
+.ie \n(.g .ds Aq \(aq
+.el       .ds Aq '
+.nh
+.ad l
+.\" Begin generated content:
+.TH "ODDMU-RELEASES" "7" "2024-02-17"
+.PP
+.SH NAME
+.PP
+oddmu-releases - what'\&s new in this releases?\&
+.PP
+.SH DESCRIPTION
+.PP
+This page lists user-visible features.\&
+.PP
+.SS 1.6 (unrelease)
+.PP
+Add \fIarchive\fR action to serve a zip file.\&
+.PP
+.SS 1.5 (2024)
+.PP
+Filtering separate sites in subdirectories via the ODDMU_FILTER environment
+variable in order to exclude them from the \fIsearch\fR action.\&
+.PP
+Add \fIversion\fR subcommand.\&
+.PP
+Add filesystem watchers to automatically reindex changed pages and reload
+changed templates.\&
+.PP
+When rendering a page, use templates in the same directory, if available.\&
+.PP
+Delete uploaded files by uploading a file with zero bytes.\&
+.PP
+.SS 1.4 (2024)
+.PP
+If stdin is a Unix-domain socket, use that to serve the site.\& Otherwise, allow
+specifying a listen address via the ODDMU_ADDRESS environment variable.\&
+.PP
+.SS 1.3 (2024)
+.PP
+Add support for resizing HEIC images (and saving them as JPG files).\&
+.PP
+.SS 1.2 (2023)
+.PP
+Add \fIlist\fR subcommand.\&
+.PP
+.SS 1.1 (2023)
+.PP
+Rewrote most of the README into man pages.\&
+.PP
+Add fediverse account rendering if ODDMU_WEBFINGER is set.\&
+.PP
+Add notifications when saving files: adding links to \fIindex\fR, \fIchanges\fR and
+\fIhashtag\fR pages.\&
+.PP
+Add \fIreplace\fR subcommand.\& Add \fImissing\fR subcommand.\& Add \fInotify\fR command.\& Add
+\fIstatic\fR command.\&
+.PP
+Add \fIdiff\fR action.\&
+.PP
+Add feed generation based on the local links from a page.\&
+.PP
+Add caching support by considering the If-Modified-Since header in requests and
+providing a Last-Modified header in responses.\&
+.PP
+Handle HEAD requests.\&
+.PP
+Remove HTML sanitization.\&
+.PP
+Remove MathJax support from the wiki parser.\& The templates never included the
+necessary MathJax JavaScript anyway so the special handling of $ was just an
+annoyance.\&
+.PP
+Drop trigram index and just search all the files.\& This takes much less RAM and
+doesn'\&t take too much time even with a few thousand pages.\&
+.PP
+Add "blog:true" and "blog:false" predicates to search.\&
+.PP
+Limit search to the current directory tree.\&
+.PP
+Do not overwrite fresh backups: there must be a 1h break before the backup is
+overwritten.\&
+.PP
+.SS 1.0 (2023)
+.PP
+Paginate search results and no longer sort search results by score.\&
+.PP
+.SS 0.9 (2023)
+.PP
+Add image resizing.\&
+.PP
+Add wiki links in double square brackets to the parser.\&
+.PP
+.SS 0.8 (2023)
+.PP
+Rename files to backups before saving.\&
+.PP
+Rename the \fIsaveUpload\fR action to \fIdrop\fR.\&
+.PP
+Add the \fIsearch\fR subcommand.\&
+.PP
+.SS 0.7 (2023)
+.PP
+Add \fIupload\fR and \fIsaveUpload\fR action so that one can upload files.\&
+.PP
+Add \fIhtml\fR subcommand.\&
+.PP
+.SS 0.6 (2003)
+.PP
+Add \fIadd\fR and \fIappend\fR action so that one can add to an existing page.\& This is
+important for me as editing pages on the phone can be cumbersome but leaving
+comments on my own site has always been easy to do.\&
+.PP
+Serve all existing files, not just text files.\&
+.PP
+Save an empty page to delete it.\&
+.PP
+Changed default permissions from 600 to 644 for files and from 700 to 755 for
+directories.\&
+.PP
+Make language detection configurable using an environment variable.\&
+.PP
+.SS 0.5 (2023)
+.PP
+Add hyphenation to templates using Peter M.\& Stahl'\&s Lingua library.\&
+.PP
+.SS 0.4 (2023)
+.PP
+Create subdirectories as necessary.\&
+.PP
+.SS 0.3 (2023)
+.PP
+Add \fIsearch\fR action using Damian Gryski'\&s trigram indexing, with scoring,
+highlighting and snippet extraction.\&
+.PP
+.SS 0.2 (2023)
+.PP
+Switch to Krzysztof Kowalczyk'\&s Go Markdown fork of Blackfriday to render
+Markdown.\& Use Dee'\&s Bluemonday to sanitize HTML.\&
+.PP
+Switch to GNU Affero GPL 3 license.\&
+.PP
+Serve text files (.\&txt).\&
+.PP
+Support serving on any port via the environment variable ODDMU_PORT.\&
+.PP
+.SS 0.1 (2015)
+.PP
+A web server that allows editing files in Wiki Creole Matt Self'\&s Cajun library.\&
+Supported actions are \fIedit\fR, \fIsave\fR, and \fIview\fR.\&
+.PP
+.SH SEE ALSO
+.PP
+\fIoddmu\fR(1)
+.PP
+.SH AUTHORS
+.PP
+Maintained by Alex Schroeder <alex@gnu.\&org>.\&

man/oddmu-releases.7.txt

@@ -0,0 +1,157 @@
+ODDMU-RELEASES(7)
+
+# NAME
+
+oddmu-releases - what's new in this releases?
+
+# DESCRIPTION
+
+This page lists user-visible features.
+
+## Next
+
+
+
+## 1.6 (2024)
+
+Add _archive_ action to serve a zip file.
+
+## 1.5 (2024)
+
+Filtering separate sites in subdirectories via the ODDMU_FILTER environment
+variable in order to exclude them from the _search_ action.
+
+Add _version_ subcommand.
+
+Add filesystem watchers to automatically reindex changed pages and reload
+changed templates.
+
+When rendering a page, use templates in the same directory, if available.
+
+Delete uploaded files by uploading a file with zero bytes.
+
+## 1.4 (2024)
+
+If stdin is a Unix-domain socket, use that to serve the site. Otherwise, allow
+specifying a listen address via the ODDMU_ADDRESS environment variable.
+
+## 1.3 (2024)
+
+Add support for resizing HEIC images (and saving them as JPG files).
+
+## 1.2 (2023)
+
+Add _list_ subcommand.
+
+## 1.1 (2023)
+
+Rewrote most of the README into man pages.
+
+Add fediverse account rendering if ODDMU_WEBFINGER is set.
+
+Add notifications when saving files: adding links to _index_, _changes_ and
+_hashtag_ pages.
+
+Add _replace_ subcommand. Add _missing_ subcommand. Add _notify_ command. Add
+_static_ command.
+
+Add _diff_ action.
+
+Add feed generation based on the local links from a page.
+
+Add caching support by considering the If-Modified-Since header in requests and
+providing a Last-Modified header in responses.
+
+Handle HEAD requests.
+
+Remove HTML sanitization.
+
+Remove MathJax support from the wiki parser. The templates never included the
+necessary MathJax JavaScript anyway so the special handling of $ was just an
+annoyance.
+
+Drop trigram index and just search all the files. This takes much less RAM and
+doesn't take too much time even with a few thousand pages.
+
+Add "blog:true" and "blog:false" predicates to search.
+
+Limit search to the current directory tree.
+
+Do not overwrite fresh backups: there must be a 1h break before the backup is
+overwritten.
+
+## 1.0 (2023)
+
+Paginate search results and no longer sort search results by score.
+
+## 0.9 (2023)
+
+Add image resizing.
+
+Add wiki links in double square brackets to the parser.
+
+## 0.8 (2023)
+
+Rename files to backups before saving.
+
+Rename the _saveUpload_ action to _drop_.
+
+Add the _search_ subcommand.
+
+## 0.7 (2023)
+
+Add _upload_ and _saveUpload_ action so that one can upload files.
+
+Add _html_ subcommand.
+
+## 0.6 (2003)
+
+Add _add_ and _append_ action so that one can add to an existing page. This is
+important for me as editing pages on the phone can be cumbersome but leaving
+comments on my own site has always been easy to do.
+
+Serve all existing files, not just text files.
+
+Save an empty page to delete it.
+
+Changed default permissions from 600 to 644 for files and from 700 to 755 for
+directories.
+
+Make language detection configurable using an environment variable.
+
+## 0.5 (2023)
+
+Add hyphenation to templates using Peter M. Stahl's Lingua library.
+
+## 0.4 (2023)
+
+Create subdirectories as necessary.
+
+## 0.3 (2023)
+
+Add _search_ action using Damian Gryski's trigram indexing, with scoring,
+highlighting and snippet extraction.
+
+## 0.2 (2023)
+
+Switch to Krzysztof Kowalczyk's Go Markdown fork of Blackfriday to render
+Markdown. Use Dee's Bluemonday to sanitize HTML.
+
+Switch to GNU Affero GPL 3 license.
+
+Serve text files (.txt).
+
+Support serving on any port via the environment variable ODDMU_PORT.
+
+## 0.1 (2015)
+
+A web server that allows editing files in Wiki Creole Matt Self's Cajun library.
+Supported actions are _edit_, _save_, and _view_.
+
+# SEE ALSO
+
+_oddmu_(1)
+
+# AUTHORS
+
+Maintained by Alex Schroeder <alex@gnu.org>.

man/oddmu-replace.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-REPLACE" "1" "2023-11-24"
+.TH "ODDMU-REPLACE" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-search.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-SEARCH" "1" "2023-12-20"
+.TH "ODDMU-SEARCH" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-search.7

@@ -1,20 +1,16 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-SEARCH" "7" "2023-10-28"
+.TH "ODDMU-SEARCH" "7" "2024-02-17"
 .PP
 .SH NAME
 .PP
 oddmu-search - understanding the Oddmu search engine
 .PP
-.SH SYNOPSIS
-.PP
-\fBoddmu search\fR \fIterms\fR.\&.\&.\&
-.PP
 .SH DESCRIPTION
 .PP
 The wiki keeps an index of all the hash tags and page titles in memory.\& Using
@@ -89,9 +85,22 @@ A document with content "This is a test" when searched with the phrase "this
 test" therefore gets a score of 8: the entire phrase does not match but each
 word gets four points.\&
 .PP
+.SH ENVIRONMENT
+.PP
+To exclude subdirectories from searches, use the ODDMU_FILTER environment
+variable.\& Set it to a regular expression matching sub-directories such as
+"^projects/".\& If search starts in a directory matching the regular expression,
+it is limited to the directory tree, as always.\& However, if search starts in a
+directory that doesn'\&t match, subdirectories that do match are skipped.\& See
+\fIoddmu-filter\fR(7).\&
+.PP
+To prevent access to a private directory tree, you must configure the web server
+in addition to setting the ODDMU_FILTER environment variable.\& See
+\fIoddmu-apache\fR(5) for more.\&
+.PP
 .SH SEE ALSO
 .PP
-\fIoddmu\fR(1), \fIoddmu-search\fR(1)
+\fIoddmu\fR(1), \fIoddmu-search\fR(1), \fIoddmu-filter\fR(7), \fIoddmu-apache\fR(5)
 .PP
 .SH AUTHORS
 .PP

man/oddmu-search.7.txt

@@ -4,10 +4,6 @@ ODDMU-SEARCH(7)
 
 oddmu-search - understanding the Oddmu search engine
 
-# SYNOPSIS
-
-*oddmu search* _terms_...
-
 # DESCRIPTION
 
 The wiki keeps an index of all the hash tags and page titles in memory. Using
@@ -69,9 +65,22 @@ A document with content "This is a test" when searched with the phrase "this
 test" therefore gets a score of 8: the entire phrase does not match but each
 word gets four points.
 
+# ENVIRONMENT
+
+To exclude subdirectories from searches, use the ODDMU_FILTER environment
+variable. Set it to a regular expression matching sub-directories such as
+"^projects/". If search starts in a directory matching the regular expression,
+it is limited to the directory tree, as always. However, if search starts in a
+directory that doesn't match, subdirectories that do match are skipped. See
+_oddmu-filter_(7).
+
+To prevent access to a private directory tree, you must configure the web server
+in addition to setting the ODDMU_FILTER environment variable. See
+_oddmu-apache_(5) for more.
+
 # SEE ALSO
 
-_oddmu_(1), _oddmu-search_(1)
+_oddmu_(1), _oddmu-search_(1), _oddmu-filter_(7), _oddmu-apache_(5)
 
 # AUTHORS
 

man/oddmu-static.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-STATIC" "1" "2024-02-09"
+.TH "ODDMU-STATIC" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu-templates.5

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-TEMPLATES" "5" "2024-02-06" "File Formats Manual"
+.TH "ODDMU-TEMPLATES" "5" "2024-02-17" "File Formats Manual"
 .PP
 .SH NAME
 .PP
@@ -99,35 +99,6 @@ For items in the feed:
 .PP
 The \fIupload.\&html\fR template cannot refer to anything.\&
 .PP
-When calling the \fIsave\fR and \fIappend\fR action, the page name is taken from the URL
-path and the page content is taken from the \fIbody\fR form parameter.\& To
-illustrate, here'\&s how to edit the "welcome" page using \fIcurl\fR:
-.PP
-.nf
-.RS 4
-curl --form body="Did you bring a towel?" 
-  http://localhost:8080/save/welcome
-.fi
-.RE
-.PP
-When calling the \fIsearch\fR action, the query is taken from the query parameter \fIq\fR.\&
-.PP
-.nf
-.RS 4
-curl \&'http://localhost:8080/search/?q=towel\&'
-.fi
-.RE
-.PP
-The page name to act upon is optionally taken from the query parameter \fIid\fR.\& In
-this case, the directory must also be part of the query parameter and not of the
-URL path.\&
-.PP
-.nf
-.RS 4
-curl \&'http://localhost:8080/view/?id=man/oddmu\&.1\&.txt\&'
-.fi
-.RE
-.PP
 .SS Non-English hyphenation
 .PP
 Automatic hyphenation by the browser requires two things: The style sheet must

man/oddmu-templates.5.txt

@@ -92,29 +92,6 @@ _{{.Date}}_, the date of the last update to this page, in RFC 822 format.
 
 The _upload.html_ template cannot refer to anything.
 
-When calling the _save_ and _append_ action, the page name is taken from the URL
-path and the page content is taken from the _body_ form parameter. To
-illustrate, here's how to edit the "welcome" page using _curl_:
-
-```
-curl --form body="Did you bring a towel?" \
-  http://localhost:8080/save/welcome
-```
-
-When calling the _search_ action, the query is taken from the query parameter _q_.
-
-```
-curl 'http://localhost:8080/search/?q=towel'
-```
-
-The page name to act upon is optionally taken from the query parameter _id_. In
-this case, the directory must also be part of the query parameter and not of the
-URL path.
-
-```
-curl 'http://localhost:8080/view/?id=man/oddmu.1.txt'
-```
-
 ## Non-English hyphenation
 
 Automatic hyphenation by the browser requires two things: The style sheet must

man/oddmu-version.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU-VERSION" "1" "2024-02-13"
+.TH "ODDMU-VERSION" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP

man/oddmu.1

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU" "1" "2024-02-13"
+.TH "ODDMU" "1" "2024-02-17"
 .PP
 .SH NAME
 .PP
@@ -19,14 +19,21 @@ Oddmu is sometimes written Oddµ because µ is the letter mu.\&
 .PP
 .SH DESCRIPTION
 .PP
-The oddmu program serves the current working directory as a wiki on port 8080.\&
-Point your browser to http://localhost:8080/ to get started.\& This is equivalent
-to http://localhost:8080/view/index – the first page you'\&ll create, most likely.\&
+Oddmu can be used as a static site generator, turning Markdown files into HTML
+files, or it can be used as a public or a private wiki server.\& If it runs as a
+public wiki server, a regular webserver should be used as reverse proxy.\&
 .PP
-If you request a page that doesn'\&t exist, oddmu tries to find a matching
+Run Oddmu without any arguments to serve the current working directory as a wiki
+on port 8080.\& Point your browser to http://localhost:8080/ to use it.\& This
+redirects you to http://localhost:8080/view/index – the first page you'\&ll
+create, most likely.\&
+.PP
+See \fIoddmu\fR(5) for details about the page formatting.\&
+.PP
+If you request a page that doesn'\&t exist, Oddmu tries to find a matching
 Markdown file by appending the extension ".\&md" to the page name.\& In the example
-above, the page name requested is "index" and the file name oddmu tries to read
-is "index.\&md".\& If no such file exists, oddmu offers you to create the page.\&
+above, the page name requested is "index" and the file name Oddmu tries to read
+is "index.\&md".\& If no such file exists, Oddmu offers you to create the page.\&
 .PP
 If your files don'\&t provide their own title ("# title"), the file name (without
 ".\&md") is used for the page title.\&
@@ -37,22 +44,126 @@ feed items are based on links in bullet lists using the asterix
 .PP
 Subdirectories are created as necessary.\&
 .PP
-See \fIoddmu\fR(5) for details about the page formatting.\&
+The wiki knows the following actions for a given page name and (optional)
+directory:
+.PP
+.PD 0
+.IP \(bu 4
+\fI/\fR redirects to /view/index
+.IP \(bu 4
+\fI/view/dir/\fR redirects to /view/dir/index
+.IP \(bu 4
+\fI/view/dir/name\fR shows a page
+.IP \(bu 4
+\fI/view/dir/name.\&md\fR shows the source text of a page
+.IP \(bu 4
+\fI/view/dir/name.\&rss\fR shows  the RSS feed for the pages linked
+.IP \(bu 4
+\fI/diff/dir/name\fR shows the last change to a page
+.IP \(bu 4
+\fI/edit/dir/name\fR shows a form to edit a page
+.IP \(bu 4
+\fI/save/dir/name\fR saves an edit
+.IP \(bu 4
+\fI/add/dir/name\fR shows a form to add to a page
+.IP \(bu 4
+\fI/append/dir/name\fR appends an addition to a page
+.IP \(bu 4
+\fI/upload/dir/name\fR shows a form to upload a file
+.IP \(bu 4
+\fI/drop/dir/name\fR saves an upload
+.IP \(bu 4
+\fI/search/dir/?\&q=term\fR to search for a term
+.IP \(bu 4
+\fI/archive/dir/name.\&zip\fR to download a zip file of a directory
+.PD
+.PP
+When calling the \fIsave\fR and \fIappend\fR action, the page name is taken from the URL
+path and the page content is taken from the \fIbody\fR form parameter.\& To
+illustrate, here'\&s how to edit the "welcome" page using \fIcurl\fR:
+.PP
+.nf
+.RS 4
+curl --form body="Did you bring a towel?" 
+  http://localhost:8080/save/welcome
+.fi
+.RE
+.PP
+When calling the \fIdrop\fR action, the query parameters used are \fIname\fR for the
+target filename, \fIfile\fR for the file to upload.\& If the query parameter
+\fImaxwidth\fR is set, an attempt is made to decode and resize the image.\& JPG, PNG
+and HEIC files can be decoded.\& Only JPG and PNG files can be encoded, however.\&
+If the target name ends in \fI.\&jpg\fR, the \fIquality\fR query parameter is also taken
+into account.\& To upload some thumbnails:
+.PP
+.nf
+.RS 4
+for f in *\&.jpg; do
+  curl --form name="$f" --form file=@"$f" --form maxwidth=100 
+    http://localhost:8080/drop/
+done
+.fi
+.RE
+.PP
+When calling the \fIsearch\fR action, the search terms are taken from the query
+parameter \fIq\fR.\&
+.PP
+.nf
+.RS 4
+curl \&'http://localhost:8080/search/?q=towel\&'
+.fi
+.RE
+.PP
+The page name to act upon is optionally taken from the query parameter \fIid\fR.\& In
+this case, the directory must also be part of the query parameter and not of the
+URL path.\&
+.PP
+.nf
+.RS 4
+curl \&'http://localhost:8080/view/?id=man/oddmu\&.1\&.txt\&'
+.fi
+.RE
+.PP
+The base name for the \fIarchive\fR action is used by the browser to save the
+downloaded file.\& For Oddmu, only the directory is important.\& The following zips
+the \fIman\fR directory and saves it as \fIman.\&zip\fR.\&
+.PP
+.nf
+.RS 4
+curl --remote-name \&'http://localhost:8080/archive/man/man\&.zip
+.fi
+.RE
 .PP
 .SH CONFIGURATION
 .PP
 The template files are the HTML files in the working directory:
-"add.\&html", "diff.\&html", "edit.\&html", "search.\&html", "upload.\&html" and
-"view.\&html".\& Please change them!\&
 .PP
-The first change you should make is to replace the name and email
-address in the footer of "view.\&html".\& Look for "Your Name" and
-"example.\&org".\&
+.PD 0
+.IP \(bu 4
+\fIview.\&html\fR shows a page
+.IP \(bu 4
+\fIdiff.\&html\fR shows the last change to a page
+.IP \(bu 4
+\fIedit.\&html\fR shows a form to edit a page
+.IP \(bu 4
+\fIadd.\&html\fR shows a form to add to a page
+.IP \(bu 4
+\fIupload.\&html\fR shows a form to upload a file
+.IP \(bu 4
+\fIsearch.\&html\fR shows the search results
+.IP \(bu 4
+\fIstatic.\&html\fR is used to generate a static site
+.IP \(bu 4
+\fIfeed.\&html\fR is used to generate a RSS feed
+.PD
 .PP
-The second change you should make is to replace the name, email
-address and domain name in "feed.\&html".\& Look for "Your Name" and
-"example.\&org".\& This second template is used to generate the RSS feeds
-(despite its ".\&html" extension).\&
+Please change the templates!\&
+.PP
+The first change you should make is to replace the name and email address in the
+footer of \fIview.\&html\fR.\& Look for "Your Name" and "example.\&org".\&
+.PP
+The second change you should make is to replace the name, email address and
+domain name in "feed.\&html".\& Look for "Your Name" and "example.\&org".\&
 .PP
 See \fIoddmu-templates\fR(5) for more.\&
 .PP
@@ -65,8 +176,12 @@ variable to either an IPv4 address or an IPv6 address.\& If ODDMU_ADDRESS is
 unset, then the program listens on all available unicast addresses, both IPv4
 and IPv6.\& Here are a few example addresses:
 .PP
-ODDMU_ADDRESS=127.\&0.\&0.\&1 # The loopback IPv4 address.\&
-ODDMU_ADDRESS=2001:db8::3:1 # An IPv6 address.\&
+.nf
+.RS 4
+ODDMU_ADDRESS=127\&.0\&.0\&.1      # The loopback IPv4 address\&.
+ODDMU_ADDRESS=2001:db8::3:1  # An IPv6 address\&.
+.fi
+.RE
 .PP
 See the Socket Activation section for an alternative method of listening which
 supports Unix-domain sockets.\&
@@ -79,9 +194,10 @@ You can enable webfinger to link fediverse accounts to their correct profile
 pages by setting ODDMU_WEBFINGER to "1".\& See \fIoddmu\fR(5).\&
 .PP
 If you use secret subdirectories, you cannot rely on the web server to hide
-those pages because search includes subdirectories.\& In order to protect those
-pages from search, you need to use the ODDMU_FILTER to a regular exression such
-as "^secret/".\& See \fIoddmu-apache\fR(5).\&
+those pages because some actions such as searching and archiving include
+subdirectories.\& They act upon a whole tree of pages, not just a single page.\& The
+ODDMU_FILTER can be used to exclude subdirectories from such tree actions.\& See
+\fIoddmu-filter\fR(7) and \fIoddmu-apache\fR(5).\&
 .PP
 .SH Socket Activation
 .PP
@@ -108,7 +224,7 @@ For an extra dose of security, consider using a Unix-domain socket.\&
 .PP
 .SH OPTIONS
 .PP
-The oddmu program can be run on the command-line using various subcommands.\&
+Oddmu can be run on the command-line using various subcommands.\&
 .PP
 .PD 0
 .IP \(bu 4
@@ -159,7 +275,7 @@ on your point of view.\& See \fIoddmu-apache\fR(5).\&
 .SH NOTES
 .PP
 Page names are filenames with ".\&md" appended.\& If your filesystem cannot handle
-it, it can'\&t be a page name.\& Filenames can contain slashes and oddmu creates
+it, it can'\&t be a page name.\& Filenames can contain slashes and Oddmu creates
 subdirectories as necessary.\&
 .PP
 Files may not end with a tilde ('\&~'\&) – these are backup files.\& When saving pages
@@ -240,12 +356,16 @@ Note that some HTML file names are special: they act as templates.\& See
 .IP \(bu 4
 \fIoddmu-apache\fR(5), on how to set up a web server such as Apache
 .IP \(bu 4
+\fIoddmu-filter\fR(7), on how to treat subdirectories as separate sites
+.IP \(bu 4
 \fIoddmu-html\fR(1), on how to render a page from the command-line
 .IP \(bu 4
 \fIoddmu-list\fR(1), on how to list pages and titles from the command-line
 .IP \(bu 4
 \fIoddmu-missing\fR(1), on how to find broken local links from the command-line
 .IP \(bu 4
+\fIoddmu-releases\fR(7), on what features are part of the latest release
+.IP \(bu 4
 \fIoddmu-replace\fR(1), on how to search and replace text from the command-line
 .IP \(bu 4
 \fIoddmu-search\fR(1), on how to run a search from the command-line

man/oddmu.1.txt

@@ -12,14 +12,21 @@ Oddmu is sometimes written Oddµ because µ is the letter mu.
 
 # DESCRIPTION
 
-The oddmu program serves the current working directory as a wiki on port 8080.
-Point your browser to http://localhost:8080/ to get started. This is equivalent
-to http://localhost:8080/view/index – the first page you'll create, most likely.
+Oddmu can be used as a static site generator, turning Markdown files into HTML
+files, or it can be used as a public or a private wiki server. If it runs as a
+public wiki server, a regular webserver should be used as reverse proxy.
 
-If you request a page that doesn't exist, oddmu tries to find a matching
+Run Oddmu without any arguments to serve the current working directory as a wiki
+on port 8080. Point your browser to http://localhost:8080/ to use it. This
+redirects you to http://localhost:8080/view/index – the first page you'll
+create, most likely.
+
+See _oddmu_(5) for details about the page formatting.
+
+If you request a page that doesn't exist, Oddmu tries to find a matching
 Markdown file by appending the extension ".md" to the page name. In the example
-above, the page name requested is "index" and the file name oddmu tries to read
-is "index.md". If no such file exists, oddmu offers you to create the page.
+above, the page name requested is "index" and the file name Oddmu tries to read
+is "index.md". If no such file exists, Oddmu offers you to create the page.
 
 If your files don't provide their own title ("# title"), the file name (without
 ".md") is used for the page title.
@@ -30,22 +37,90 @@ feed items are based on links in bullet lists using the asterix
 
 Subdirectories are created as necessary.
 
-See _oddmu_(5) for details about the page formatting.
+The wiki knows the following actions for a given page name and (optional)
+directory:
+
+- _/_ redirects to /view/index
+- _/view/dir/_ redirects to /view/dir/index
+- _/view/dir/name_ shows a page
+- _/view/dir/name.md_ shows the source text of a page
+- _/view/dir/name.rss_ shows  the RSS feed for the pages linked
+- _/diff/dir/name_ shows the last change to a page
+- _/edit/dir/name_ shows a form to edit a page
+- _/save/dir/name_ saves an edit
+- _/add/dir/name_ shows a form to add to a page
+- _/append/dir/name_ appends an addition to a page
+- _/upload/dir/name_ shows a form to upload a file
+- _/drop/dir/name_ saves an upload
+- _/search/dir/?q=term_ to search for a term
+- _/archive/dir/name.zip_ to download a zip file of a directory
+
+When calling the _save_ and _append_ action, the page name is taken from the URL
+path and the page content is taken from the _body_ form parameter. To
+illustrate, here's how to edit the "welcome" page using _curl_:
+
+```
+curl --form body="Did you bring a towel?" \
+  http://localhost:8080/save/welcome
+```
+
+When calling the _drop_ action, the query parameters used are _name_ for the
+target filename, _file_ for the file to upload. If the query parameter
+_maxwidth_ is set, an attempt is made to decode and resize the image. JPG, PNG
+and HEIC files can be decoded. Only JPG and PNG files can be encoded, however.
+If the target name ends in _.jpg_, the _quality_ query parameter is also taken
+into account. To upload some thumbnails:
+
+```
+for f in *.jpg; do
+  curl --form name="$f" --form file=@"$f" --form maxwidth=100 \
+    http://localhost:8080/drop/
+done
+```
+
+When calling the _search_ action, the search terms are taken from the query
+parameter _q_.
+
+```
+curl 'http://localhost:8080/search/?q=towel'
+```
+
+The page name to act upon is optionally taken from the query parameter _id_. In
+this case, the directory must also be part of the query parameter and not of the
+URL path.
+
+```
+curl 'http://localhost:8080/view/?id=man/oddmu.1.txt'
+```
+
+The base name for the _archive_ action is used by the browser to save the
+downloaded file. For Oddmu, only the directory is important. The following zips
+the _man_ directory and saves it as _man.zip_.
+
+```
+curl --remote-name 'http://localhost:8080/archive/man/man.zip
+```
 
 # CONFIGURATION
 
 The template files are the HTML files in the working directory:
-"add.html", "diff.html", "edit.html", "search.html", "upload.html" and
-"view.html". Please change them!
 
-The first change you should make is to replace the name and email
-address in the footer of "view.html". Look for "Your Name" and
-"example.org".
+- _view.html_ shows a page
+- _diff.html_ shows the last change to a page
+- _edit.html_ shows a form to edit a page
+- _add.html_ shows a form to add to a page
+- _upload.html_ shows a form to upload a file
+- _search.html_ shows the search results
+- _static.html_ is used to generate a static site
+- _feed.html_ is used to generate a RSS feed
 
-The second change you should make is to replace the name, email
-address and domain name in "feed.html". Look for "Your Name" and
-"example.org". This second template is used to generate the RSS feeds
-(despite its ".html" extension).
+Please change the templates!
+
+The first change you should make is to replace the name and email address in the
+footer of _view.html_. Look for "Your Name" and "example.org".
+
+The second change you should make is to replace the name, email address and
+domain name in "feed.html". Look for "Your Name" and "example.org".
 
 See _oddmu-templates_(5) for more.
 
@@ -58,8 +133,10 @@ variable to either an IPv4 address or an IPv6 address. If ODDMU_ADDRESS is
 unset, then the program listens on all available unicast addresses, both IPv4
 and IPv6. Here are a few example addresses:
 
-ODDMU_ADDRESS=127.0.0.1 # The loopback IPv4 address.
-ODDMU_ADDRESS=2001:db8::3:1 # An IPv6 address.
+```
+ODDMU_ADDRESS=127.0.0.1      # The loopback IPv4 address.
+ODDMU_ADDRESS=2001:db8::3:1  # An IPv6 address.
+```
 
 See the Socket Activation section for an alternative method of listening which
 supports Unix-domain sockets.
@@ -72,9 +149,10 @@ You can enable webfinger to link fediverse accounts to their correct profile
 pages by setting ODDMU_WEBFINGER to "1". See _oddmu_(5).
 
 If you use secret subdirectories, you cannot rely on the web server to hide
-those pages because search includes subdirectories. In order to protect those
-pages from search, you need to use the ODDMU_FILTER to a regular exression such
-as "^secret/". See _oddmu-apache_(5).
+those pages because some actions such as searching and archiving include
+subdirectories. They act upon a whole tree of pages, not just a single page. The
+ODDMU_FILTER can be used to exclude subdirectories from such tree actions. See
+_oddmu-filter_(7) and _oddmu-apache_(5).
 
 # Socket Activation
 
@@ -101,7 +179,7 @@ For an extra dose of security, consider using a Unix-domain socket.
 
 # OPTIONS
 
-The oddmu program can be run on the command-line using various subcommands.
+Oddmu can be run on the command-line using various subcommands.
 
 - to generate the HTML for a single page, see _oddmu-html_(1)
 - to generate the HTML for the entire site, using Oddmu as a static site
@@ -142,7 +220,7 @@ on your point of view. See _oddmu-apache_(5).
 # NOTES
 
 Page names are filenames with ".md" appended. If your filesystem cannot handle
-it, it can't be a page name. Filenames can contain slashes and oddmu creates
+it, it can't be a page name. Filenames can contain slashes and Oddmu creates
 subdirectories as necessary.
 
 Files may not end with a tilde ('~') – these are backup files. When saving pages
@@ -218,9 +296,11 @@ _oddmu-templates_(5) for their names and their use.
 - _oddmu_(5), about the markup syntax and how feeds are generated based on link lists
 - _oddmu.service_(5), on how to run the service under systemd
 - _oddmu-apache_(5), on how to set up a web server such as Apache
+- _oddmu-filter_(7), on how to treat subdirectories as separate sites
 - _oddmu-html_(1), on how to render a page from the command-line
 - _oddmu-list_(1), on how to list pages and titles from the command-line
 - _oddmu-missing_(1), on how to find broken local links from the command-line
+- _oddmu-releases_(7), on what features are part of the latest release
 - _oddmu-replace_(1), on how to search and replace text from the command-line
 - _oddmu-search_(1), on how to run a search from the command-line
 - _oddmu-search_(7), on how search works

man/oddmu.5

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU" "5" "2023-11-12" "File Formats Manual"
+.TH "ODDMU" "5" "2024-02-17" "File Formats Manual"
 .PP
 .SH NAME
 .PP

man/oddmu.service.5

@@ -1,11 +1,11 @@
-.\" Generated by scdoc 1.11.2
+.\" Generated by scdoc 1.11.3
 .\" Complete documentation for this program is not available as a GNU info page
 .ie \n(.g .ds Aq \(aq
 .el       .ds Aq '
 .nh
 .ad l
 .\" Begin generated content:
-.TH "ODDMU.SERVICE" "5" "2024-01-17"
+.TH "ODDMU.SERVICE" "5" "2024-02-17"
 .PP
 .SH NAME
 .PP

man_test.go

@@ -0,0 +1,31 @@
+package main
+
+import (
+	"github.com/stretchr/testify/assert"
+	"io/fs"
+	"path/filepath"
+	"os"
+	"strings"
+	"testing"
+)
+
+func TestManPages(t *testing.T) {
+	b, err := os.ReadFile("man/oddmu.1.txt");
+	main := string(b)
+	assert.NoError(t, err)
+	filepath.Walk("man", func (path string, info fs.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		if strings.HasSuffix(path, ".txt") &&
+			path != "man/oddmu.1.txt" {
+			s := strings.TrimPrefix(path, "man/")
+			s = strings.TrimSuffix(s, ".txt")
+			i := strings.LastIndex(s, ".")
+			ref := "_" + s[:i] + "_(" + s[i+1:] + ")"
+			assert.Contains(t, main, ref, ref)
+		}
+		return nil
+	})
+}
+

search.go

@@ -122,22 +122,21 @@ func search(q, dir, filter string, page int, all bool) ([]*Page, bool) {
 // The regular expression can be used to ensure that search does not descend into subdirectories unless the search
 // already starts there. Given the pages a, public/b and secret/c and ODDMU_FILTER=^secret/ then if search starts in the
 // root directory /, search does not enter secret/, but if search starts in secret/, search does search the pages in
-// secret/ – it us up to the web server to ensure access to secret/ is limited. More specifically: If the current
-// directory matches the regular expression, the page names must match; if the regular expression does not match the
-// current directory, the page name must not match the filter either. If the filter is empty, all prefixes and all page
-// names match, so no problem.
+// secret/ – it us up to the web server to ensure access to secret/ is limited. More specifically: the page names must
+// match the prefix, always; if prefix also matches the filter, this means the page names are all part of a "separate
+// site"; if the prefix does not match the filter, then the page names must also not match the filter since only the
+// "main site" is shown. If the filter is empty, all prefixes and all page names match, so no problem.
 func filterPath(names []string, prefix, filter string) []string {
 	re, err := regexp.Compile(filter)
 	if err != nil {
 		log.Println("ODDMU_FILTER does not compile:", filter, err)
 		return []string{}
 	}
-	mustMatch := re.MatchString(prefix)
+	matches := re.MatchString(prefix)
 	r := make([]string, 0)
 	for _, name := range names {
 		if strings.HasPrefix(name, prefix) &&
-			(mustMatch && re.MatchString(name) ||
-				!mustMatch && !re.MatchString(name)) {
+			(matches || !re.MatchString(name)) {
 			r = append(r, name)
 		}
 	}

view.html

@@ -23,6 +23,7 @@ img { max-width: 100%; }
       <a href="/edit/{{.Name}}" accesskey="e">Edit</a>
       <a href="/add/{{.Name}}" accesskey="a">Add</a>
       <a href="/diff/{{.Name}}" accesskey="d">Diff</a>
+      <a href="/archive/{{.Dir}}data.zip" accesskey="z">Zip</a>
       <a href="/upload/{{.Dir}}?filename={{.Base}}-1.jpg" accesskey="u">Upload</a>
       <form role="search" action="/search/{{.Dir}}" method="GET">
         <label for="search">Search:</label>

wiki.go

@@ -141,6 +141,7 @@ func scheduleInstallWatcher() {
 
 func serve() {
 	http.HandleFunc("/", rootHandler)
+	http.HandleFunc("/archive/", makeHandler(archiveHandler, true))
 	http.HandleFunc("/view/", makeHandler(viewHandler, true))
 	http.HandleFunc("/diff/", makeHandler(diffHandler, true))
 	http.HandleFunc("/edit/", makeHandler(editHandler, true))