13 Commits

Author SHA1 Message Date
Alex Schroeder
faf9198edd Add sitemap command and handler 2026-01-02 22:30:02 +01:00
Alex Schroeder
e9666a5ec5 Fixed missing command
Outside of the test it always got an empty index and therefore never
found any missing pages.
2026-01-02 14:33:42 +01:00
Alex Schroeder
f71a0e9780 Feeds page through yearly archives, too 2026-01-01 17:44:44 +01:00
Alex Schroeder
aca1d82fe0 Feed subcommand uses page name to determine date
The pubDate of the items in an archive feed should not be based on the
last modified time of the file because mass edits are irrelevant in
this context. The intended creation time is more important.
2025-12-31 20:28:41 +01:00
Alex Schroeder
290ad16e09 Change {{.Html}} to {{.HTML}} in the themes folder
Add note to the release page.
2025-12-06 14:24:19 +01:00
Alex Schroeder
ad1732f57f Fix all gocritic and golint issues 2025-12-06 14:10:20 +01:00
Alex Schroeder
67120af7cc Run goimports and rearrange imports 2025-12-06 13:38:46 +01:00
Alex Schroeder
c186253a25 Add check and fix targets to Makefile 2025-12-06 13:35:34 +01:00
Alex Schroeder
d0d4545f74 Ignore some errors when generating a static site
When using the static subcommand:

- ignore existing directory;
- remove existing files before creating a link;
- report any remaining errors.
2025-12-06 13:35:31 +01:00
Alex Schroeder
c32f087af4 Add shrink option to static command 2025-12-05 14:22:21 +01:00
Alex Schroeder
f657ac60a3 Extract MIME type detection by extension 2025-12-04 23:33:46 +01:00
Alex Schroeder
16ae6cc143 Don't load the index for notifyCli
It's not required for the command-line and improves execution time.
2025-11-08 23:33:33 +01:00
Alex Schroeder
e041c5ecae Fiddle with the wording of the README 2025-09-29 16:53:25 +02:00
112 changed files with 1058 additions and 400 deletions

View File

@@ -10,6 +10,10 @@ help:
@echo " runs program, offline"
@echo make test
@echo " runs the tests without log output"
@echo make check
@echo " checks the code with golint and gocritic"
@echo make fix
@echo " fixes formatting issues with goimports instead of go fmt"
@echo make docs
@echo " create man pages from text files"
@echo make build
@@ -34,6 +38,14 @@ test:
rm -rf testdata/*
go test -shuffle on .
check:
golint
gocritic check
fix:
goimports -w *.go
run:
go run .

View File

@@ -8,20 +8,18 @@ with Markdown files, turning them into HTML files. HTML templates
allow the customisation of headers, footers and styling. There are no
plugins.
Oddμ is well suited as a self-hosted, single-user web application,
when there is no need for collaboration on the site itself. Links and
email connect you to the rest of the net. The wiki can be public or
private.
Oddμ is well suited as a self-hosted, single-user web application.
Edit the pages from your phone or laptop, while you're on the move.
If the site is public, use a regular web server as a proxy to make
people log in before making changes. As there is no version history,
it is not possible to undo vandalism and spam. Only grant write-access
it is not easy to undo vandalism and spam. Only grant write-access
to people you trust.
If the site is private, running on a local machine and unreachable
from the Internet, no such precautions are necessary.
Oddμ is well suited as a secondary medium for a close-knit group:
This makes Oddμ well suited as a secondary medium for a close-knit group:
collaboration and conversation happens elsewhere, in chat, on social
media. The wiki serves as the text repository that results from these
discussions.
@@ -43,7 +41,7 @@ Other files can be uploaded and images (ending in `.jpg`, `.jpeg`,
## Documentation
This project uses man(1) pages. They are generated from text files
This project uses man pages. They are generated from text files
using [scdoc](https://git.sr.ht/~sircmpwn/scdoc). These are the files
available:
@@ -112,6 +110,10 @@ Markdown pages from the command line.
This man page documents the "feed" subcommand to generate a feed from
Markdown pages from the command line.
[oddmu-sitemap(1)](https://alexschroeder.ch/view/oddmu/oddmu-sitemap.1):
This man page documents the "sitemap" subcommand to generate the
static sitemap from the command line.
[oddmu-static(1)](https://alexschroeder.ch/view/oddmu/oddmu-static.1):
This man page documents the "static" subcommand to generate an entire
static website from the command line, avoiding the need to run Oddmu
@@ -257,6 +259,7 @@ high-level introduction to the various source files.
- `preview.go` implements the `/preview` handler
- `score.go` implements the page scoring when showing search results
- `search.go` implements the `/search` handler
- `sitemap.go` implements the `/sitemap` handler
- `snippets.go` implements the page summaries for search results
- `templates.go` implements template loading and reloading
- `tokenizer.go` implements the various tokenizers used

View File

@@ -2,13 +2,14 @@ package main
import (
"encoding/json"
"github.com/gomarkdown/markdown/ast"
"github.com/gomarkdown/markdown/parser"
"io"
"log"
"net/http"
"os"
"sync"
"github.com/gomarkdown/markdown/ast"
"github.com/gomarkdown/markdown/parser"
)
// useWebfinger indicates whether Oddmu looks up the profile pages of fediverse accounts. To enable this, set the
@@ -38,7 +39,7 @@ func init() {
// accountLink links a social media accountLink like @accountLink@domain to a profile page like https://domain/user/accountLink. Any
// accountLink seen for the first time uses a best guess profile URI. It is also looked up using webfinger, in parallel. See
// lookUpAccountUri. If the lookup succeeds, the best guess is replaced with the new URI so on subsequent requests, the
// lookUpAccountURI. If the lookup succeeds, the best guess is replaced with the new URI so on subsequent requests, the
// URI is correct.
func accountLink(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
data = data[offset:]
@@ -56,9 +57,8 @@ func accountLink(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
if d != 0 {
// more than one @ is invalid
return 0, nil
} else {
d = i + 1 // skip @ of domain
}
d = i + 1 // skip @ of domain
}
i++
}
@@ -79,7 +79,7 @@ func accountLink(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
log.Printf("Looking up %s\n", account)
uri = "https://" + string(domain) + "/users/" + string(user[1:])
accounts.uris[string(account)] = uri // prevent more lookings
go lookUpAccountUri(string(account), string(domain))
go lookUpAccountURI(string(account), string(domain))
}
link := &ast.Link{
AdditionalAttributes: []string{`class="account"`},
@@ -90,9 +90,9 @@ func accountLink(p *parser.Parser, data []byte, offset int) (int, ast.Node) {
return i, link
}
// lookUpAccountUri is called for accounts that haven't been seen before. It calls webfinger and parses the JSON. If
// lookUpAccountURI is called for accounts that haven't been seen before. It calls webfinger and parses the JSON. If
// possible, it extracts the link to the profile page and replaces the entry in accounts.
func lookUpAccountUri(account, domain string) {
func lookUpAccountURI(account, domain string) {
uri := "https://" + domain + "/.well-known/webfinger"
resp, err := http.Get(uri + "?resource=acct:" + account)
if err != nil {

View File

@@ -1,8 +1,9 @@
package main
import (
"github.com/stretchr/testify/assert"
"testing"
"github.com/stretchr/testify/assert"
)
func TestWebfingerParsing(t *testing.T) {

View File

@@ -53,10 +53,12 @@ func appendHandler(w http.ResponseWriter, r *http.Request, name string) {
func (p *Page) append(body []byte) {
// ensure an empty line at the end
if bytes.HasSuffix(p.Body, []byte("\n\n")) {
} else if bytes.HasSuffix(p.Body, []byte("\n")) {
switch {
case bytes.HasSuffix(p.Body, []byte("\n\n")):
// two newlines, nothing to add
case bytes.HasSuffix(p.Body, []byte("\n")):
p.Body = append(p.Body, '\n')
} else {
default:
p.Body = append(p.Body, '\n', '\n')
}
p.Body = append(p.Body, body...)

View File

@@ -1,13 +1,14 @@
package main
import (
"github.com/stretchr/testify/assert"
"net/http"
"net/url"
"os"
"regexp"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestEmptyLineAdd(t *testing.T) {

View File

@@ -2,11 +2,12 @@ package main
import (
"archive/zip"
"github.com/stretchr/testify/assert"
"net/http"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestArchive(t *testing.T) {

View File

@@ -38,7 +38,7 @@ func (p *Page) notify() error {
return err
}
}
p.renderHtml() // to set hashtags
p.renderHTML() // to set hashtags
for _, hashtag := range p.Hashtags {
err := addLink(path.Join(dir, hashtag), false, link, re)
if err != nil {
@@ -98,12 +98,13 @@ func addLinkWithDate(name, link string, re *regexp.Regexp) error {
if loc[0] >= 14 {
re := regexp.MustCompile(`(?m)^## (\d\d\d\d-\d\d-\d\d)\n`)
m := re.Find(p.Body[loc[0]-14 : loc[0]])
if m == nil {
switch {
case m == nil:
// not a date: insert date, don't move insertion point
} else if string(p.Body[loc[0]-11:loc[0]-1]) == date {
case string(p.Body[loc[0]-11:loc[0]-1]) == date:
// if the date is our date, don't add it, don't move insertion point
addDate = false
} else {
default:
// if the date is not out date, move the insertion point
loc[0] -= 14
}
@@ -148,10 +149,9 @@ func addLink(name string, mandatory bool, link string, re *regexp.Regexp) error
if mandatory {
p = &Page{Name: name, Body: []byte(link)}
return p.save()
} else {
// Skip non-existing files: no error
return nil
}
// Skip non-existing files: no error
return nil
}
org := string(p.Body)
addLinkToPage(p, link, re)

View File

@@ -1,11 +1,12 @@
package main
import (
"github.com/stretchr/testify/assert"
"os"
"regexp"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
// Note TestEditSaveChanges and TestAddAppendChanges.
@@ -119,9 +120,9 @@ func TestChangesWithList(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// new line was added at the beginning of the list
assert.Equal(t, intro+d+new_line+line, string(s))
assert.Equal(t, intro+d+newLine+line, string(s))
}
func TestChangesWithOldList(t *testing.T) {
@@ -136,9 +137,9 @@ func TestChangesWithOldList(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// new line was added at the beginning of the list
assert.Equal(t, intro+d+new_line+"\n"+y+line, string(s))
assert.Equal(t, intro+d+newLine+"\n"+y+line, string(s))
}
func TestChangesWithOldDisappearingListAtTheEnd(t *testing.T) {
@@ -153,9 +154,9 @@ func TestChangesWithOldDisappearingListAtTheEnd(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// new line was added at the beginning of the list, with the new date, and the old date disappeared
assert.Equal(t, intro+d+new_line, string(s))
assert.Equal(t, intro+d+newLine, string(s))
}
func TestChangesWithOldDisappearingListInTheMiddle(t *testing.T) {
@@ -172,9 +173,9 @@ func TestChangesWithOldDisappearingListInTheMiddle(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// new line was added at the beginning of the list, with the new date, and the old date disappeared
assert.Equal(t, intro+d+new_line+"\n"+yy+other, string(s))
assert.Equal(t, intro+d+newLine+"\n"+yy+other, string(s))
}
func TestChangesWithListAtTheTop(t *testing.T) {
@@ -187,9 +188,9 @@ func TestChangesWithListAtTheTop(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// new line was added at the top, no error due to missing introduction
assert.Equal(t, d+new_line+line, string(s))
assert.Equal(t, d+newLine+line, string(s))
}
func TestChangesWithNoList(t *testing.T) {
@@ -202,9 +203,9 @@ func TestChangesWithNoList(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// into is still there and a new list was started
assert.Equal(t, intro+"\n\n"+d+new_line, string(s))
assert.Equal(t, intro+"\n\n"+d+newLine, string(s))
}
func TestChangesWithUpdate(t *testing.T) {
@@ -219,9 +220,9 @@ func TestChangesWithUpdate(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// the change was already listed, but now it moved up and has a new title
assert.Equal(t, intro+d+new_line+other, string(s))
assert.Equal(t, intro+d+newLine+other, string(s))
}
func TestChangesWithNoChangeToTheOrder(t *testing.T) {
@@ -236,9 +237,9 @@ func TestChangesWithNoChangeToTheOrder(t *testing.T) {
p.notify()
s, err := os.ReadFile("testdata/changes/changes.md")
assert.NoError(t, err)
new_line := "* [testdata/changes/alex](alex)\n"
newLine := "* [testdata/changes/alex](alex)\n"
// the change was already listed at the top, so just use the new title
assert.Equal(t, intro+d+new_line+other, string(s))
assert.Equal(t, intro+d+newLine+other, string(s))
// since the file has changed, a backup was necessary
assert.FileExists(t, "testdata/changes/changes.md~")
}

View File

@@ -2,13 +2,14 @@ package main
import (
"bytes"
"github.com/sergi/go-diff/diffmatchpatch"
"html"
"html/template"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/sergi/go-diff/diffmatchpatch"
)
func diffHandler(w http.ResponseWriter, r *http.Request, name string) {
@@ -18,11 +19,11 @@ func diffHandler(w http.ResponseWriter, r *http.Request, name string) {
return
}
p.handleTitle(true)
p.renderHtml()
p.renderHTML()
renderTemplate(w, p.Dir(), "diff", p)
}
// Diff computes the diff for a page. At this point, renderHtml has already been called so the Name is escaped.
// Diff computes the diff for a page. At this point, renderHTML has already been called so the Name is escaped.
func (p *Page) Diff() template.HTML {
fp := filepath.FromSlash(p.Name)
a := fp + ".md~"

View File

@@ -1,11 +1,12 @@
package main
import (
"github.com/stretchr/testify/assert"
"net/http"
"os"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestDiff(t *testing.T) {

View File

@@ -1,12 +1,13 @@
package main
import (
"github.com/stretchr/testify/assert"
"net/http"
"net/url"
"os"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestEditSave(t *testing.T) {

View File

@@ -4,13 +4,14 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
htmlTemplate "html/template"
"io"
"os"
"strings"
textTemplate "text/template"
"time"
"github.com/google/subcommands"
)
type exportCmd struct {
@@ -62,7 +63,7 @@ func exportCli(w io.Writer, templateName string, idx *indexStore) subcommands.Ex
return subcommands.ExitFailure
}
p.handleTitle(false)
p.renderHtml()
p.renderHTML()
fi, err := os.Stat(name + ".md")
if err != nil {
fmt.Fprintf(os.Stderr, "Stat %s: %s\n", name, err)
@@ -72,7 +73,7 @@ func exportCli(w io.Writer, templateName string, idx *indexStore) subcommands.Ex
it.Title = p.Title
it.Name = p.Name
it.Body = p.Body
it.Html = htmlTemplate.HTML(htmlTemplate.HTMLEscaper(p.Html))
it.HTML = htmlTemplate.HTML(htmlTemplate.HTMLEscaper(p.HTML))
it.Hashtags = p.Hashtags
items = append(items, it)
}

View File

@@ -2,11 +2,12 @@ package main
import (
"bytes"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"os"
"regexp"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
)
func TestExportCmd(t *testing.T) {
@@ -40,7 +41,7 @@ func TestExportCmdJsonFeed(t *testing.T) {
"title": "{{.Title}}",
"language": "{{.Language}}"
"date_modified": "{{.Date}}",
"content_html": "{{.Html}}",
"content_html": "{{.HTML}}",
"tags": [{{range .Hashtags}}"{{.}}",{{end}}""],
},{{end}}
{}

83
feed.go
View File

@@ -2,13 +2,22 @@ package main
import (
"bytes"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"html/template"
"os"
"path"
"path/filepath"
"strconv"
"time"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
)
type dateSource int
const (
ModTime dateSource = iota
URL
)
// Item is a Page plus a Date.
@@ -36,17 +45,22 @@ type Feed struct {
Items []Item
// From is where the item number where the feed starts. It defaults to 0. Prev and From are the item numbers of
// the previous and the next page of the feed. N is the number of items per page.
// The previous and the next page of the feed. N is the number of items per page. Next goes further into the
// past.
Prev, Next, From, N int
// When paging through the index or year pages, link to the next or previous years. NextYear goes further into
// the past (is smaller).
PrevYear, NextYear int
// Complete is set when there is no pagination.
Complete bool
}
// feed returns a RSS 2.0 feed for any page. The feed items it contains are the pages linked from in list items starting
// with an asterisk ("*"). The feed starts from a certain item and contains n items. If n is 0, the feed is complete
// (unpaginated).
func feed(p *Page, ti time.Time, from, n int) *Feed {
// (unpaginated). The
func feed(p *Page, ti time.Time, from, n int, source dateSource) *Feed {
feed := new(Feed)
feed.Name = p.Name
feed.Title = p.Title
@@ -57,6 +71,11 @@ func feed(p *Page, ti time.Time, from, n int) *Feed {
feed.Complete = true
} else if from > n {
feed.Prev = from - n
} else {
year, err := p.BlogYear()
if err == nil && p.ArchiveExists(year+1) {
feed.PrevYear = year + 1
}
}
to := from + n
parser, _ := wikiParser()
@@ -92,24 +111,64 @@ func feed(p *Page, ti time.Time, from, n int) *Feed {
}
// i counts links, not actual existing pages
name := path.Join(p.Dir(), string(link.Destination))
fi, err := os.Stat(filepath.FromSlash(name) + ".md")
if err != nil {
return ast.GoToNext
}
p2, err := loadPage(name)
if err != nil {
return ast.GoToNext
}
p2.handleTitle(false)
p2.renderHtml()
it := Item{Date: fi.ModTime().Format(time.RFC1123Z)}
p2.renderHTML()
date, err := p2.Date(source)
if err != nil {
return ast.GoToNext
}
it := Item{Date: date.Format(time.RFC1123Z)}
it.Title = p2.Title
it.Name = p2.Name
it.Html = template.HTML(template.HTMLEscaper(p2.Html))
it.HTML = template.HTML(template.HTMLEscaper(p2.HTML))
it.Hashtags = p2.Hashtags
items = append(items, it)
return ast.GoToNext
})
// If there are no more "next" links but there is a next page, add it.
if feed.Next == 0 {
year, err := p.BlogYear()
if err == nil && p.ArchiveExists(year-1) {
feed.NextYear = year - 1
}
}
feed.Items = items
return feed
}
// Date returns the page's last modification date if the data source is ModTime. If the data source is URL, then the
// first 10 characters are parsed as an ISO date string and the time returned is for that date, 0:00, UTC.
func (p *Page) Date(source dateSource) (time.Time, error) {
if source == URL && p.IsBlog() {
name := path.Base(p.Name)
return time.Parse(time.DateOnly, name[0:10])
}
return p.ModTime()
}
// BLogYear returns the current year if the page name is "index". If the page name is a number such as "2026" then
// this is parsed as an integer and returned.
func (p *Page) BlogYear() (int, error) {
name := path.Base(p.Name)
if name == "index" {
return time.Now().Year(), nil
}
ui, err := strconv.ParseUint(name, 10, 16)
if err == nil {
return int(ui), nil
}
return 0, err
}
// ArchiveExists returns true if a page exists in the same directory as the current one with a page name matching
// the year given.
func (p *Page) ArchiveExists(year int) bool {
name := path.Join(p.Dir(), strconv.Itoa(year))
fp := filepath.FromSlash(name) + ".md"
_, err := os.Stat(fp)
return err == nil
}

View File

@@ -7,8 +7,10 @@
<managingEditor>you@example.org (Your Name)</managingEditor>
<webMaster>you@example.org (Your Name)</webMaster>
<atom:link href="https://example.org/view/{{.Path}}.rss" rel="self" type="application/rss+xml"/>{{if .From}}
<atom:link href="https://example.org/view/{{.Path}}.rss?from={{.Prev}}&amp;n={{.N}}" rel="previous" type="application/rss+xml"/>{{end}}{{if .Next}}
<atom:link href="https://example.org/view/{{.Path}}.rss?from={{.Next}}&amp;n={{.N}}" rel="next" type="application/rss+xml"/>{{end}}{{if .Complete}}
<atom:link href="https://example.org/view/{{.Path}}.rss?from={{.Prev}}&amp;n={{.N}}" rel="previous" type="application/rss+xml"/>{{end}}{{if .PrevYear}}
<atom:link href="https://example.org/view/{{.Dir}}{{.PrevYear}}.rss?n={{.N}}" rel="previous" type="application/rss+xml"/>{{end}}{{if .Next}}
<atom:link href="https://example.org/view/{{.Path}}.rss?from={{.Next}}&amp;n={{.N}}" rel="next" type="application/rss+xml"/>{{end}}{{if .NextYear}}
<atom:link href="https://example.org/view/{{.Dir}}{{.NextYear}}.rss?n={{.N}}" rel="next" type="application/rss+xml"/>{{end}}{{if .Complete}}
<fh:complete/>{{end}}
<description>This is the digital garden of Your Name.</description>
<image>
@@ -21,7 +23,7 @@
<title>{{.Title}}</title>
<link>https://example.org/view/{{.Path}}</link>
<guid>https://example.org/view/{{.Path}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -4,11 +4,12 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"io"
"os"
"strings"
"time"
"github.com/google/subcommands"
)
type feedCmd struct {
@@ -67,7 +68,7 @@ func feedCli(w io.Writer, args []string) subcommands.ExitStatus {
// printFeed prints the complete feed for a page (unpaginated).
func (p *Page) printFeed(w io.Writer, ti time.Time) subcommands.ExitStatus {
f := feed(p, ti, 0, 0)
f := feed(p, ti, 0, 0, URL)
if len(f.Items) == 0 {
fmt.Fprintf(os.Stderr, "Empty feed for %s\n", p.Name)
return subcommands.ExitFailure

View File

@@ -2,16 +2,18 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestFeedCmd(t *testing.T) {
cleanup(t, "testdata/complete")
p := &Page{Name: "testdata/complete/one", Body: []byte("# One\n")}; p.save()
p := &Page{Name: "testdata/complete/2025-12-01", Body: []byte("# 2025-12-01\n")}
p.save()
p = &Page{Name: "testdata/complete/index", Body: []byte(`# Index
* [one](one)
* [2025-12-01](2025-12-01)
`)}
p.save()
@@ -19,4 +21,6 @@ func TestFeedCmd(t *testing.T) {
s := feedCli(b, []string{"testdata/complete/index.md"})
assert.Equal(t, subcommands.ExitSuccess, s)
assert.Contains(t, b.String(), "<fh:complete/>")
assert.Contains(t, b.String(), "<title>2025-12-01</title>")
assert.Contains(t, b.String(), "<pubDate>Mon, 01 Dec 2025 00:00:00") // ignore timezone
}

View File

@@ -1,10 +1,12 @@
package main
import (
"github.com/stretchr/testify/assert"
"fmt"
"net/http"
"testing"
"net/url"
"testing"
"github.com/stretchr/testify/assert"
)
func TestFeed(t *testing.T) {
@@ -54,20 +56,29 @@ Writing poems about plants.
assert.Contains(t, body, "<category>Palmtree</category>")
}
func TestFeedPagination(t *testing.T) {
cleanup(t, "testdata/pagination")
p := &Page{Name: "testdata/pagination/one", Body: []byte("# One\n")}; p.save()
p = &Page{Name: "testdata/pagination/two", Body: []byte("# Two\n")}; p.save()
p = &Page{Name: "testdata/pagination/three", Body: []byte("# Three\n")}; p.save()
p = &Page{Name: "testdata/pagination/four", Body: []byte("# Four\n")}; p.save()
p = &Page{Name: "testdata/pagination/five", Body: []byte("# Five\n")}; p.save()
p = &Page{Name: "testdata/pagination/six", Body: []byte("# Six\n")}; p.save()
p = &Page{Name: "testdata/pagination/seven", Body: []byte("# Seven\n")}; p.save()
p = &Page{Name: "testdata/pagination/eight", Body: []byte("# Eight\n")}; p.save()
p = &Page{Name: "testdata/pagination/nine", Body: []byte("# Nine\n")}; p.save()
p = &Page{Name: "testdata/pagination/ten", Body: []byte("# Ten\n")}; p.save()
p := &Page{Name: "testdata/pagination/one", Body: []byte("# One\n")}
p.save()
p = &Page{Name: "testdata/pagination/two", Body: []byte("# Two\n")}
p.save()
p = &Page{Name: "testdata/pagination/three", Body: []byte("# Three\n")}
p.save()
p = &Page{Name: "testdata/pagination/four", Body: []byte("# Four\n")}
p.save()
p = &Page{Name: "testdata/pagination/five", Body: []byte("# Five\n")}
p.save()
p = &Page{Name: "testdata/pagination/six", Body: []byte("# Six\n")}
p.save()
p = &Page{Name: "testdata/pagination/seven", Body: []byte("# Seven\n")}
p.save()
p = &Page{Name: "testdata/pagination/eight", Body: []byte("# Eight\n")}
p.save()
p = &Page{Name: "testdata/pagination/nine", Body: []byte("# Nine\n")}
p.save()
p = &Page{Name: "testdata/pagination/ten", Body: []byte("# Ten\n")}
p.save()
p = &Page{Name: "testdata/pagination/index", Body: []byte(`# Index
* [one](one)
@@ -88,7 +99,8 @@ func TestFeedPagination(t *testing.T) {
assert.Contains(t, body, "<title>Ten</title>")
assert.NotContains(t, body, `<atom:link href="https://example.org/view/testdata/pagination/index.rss?from=10&n=10" rel="next" type="application/rss+xml"/>`)
p = &Page{Name: "testdata/pagination/eleven", Body: []byte("# Eleven\n")}; p.save()
p = &Page{Name: "testdata/pagination/eleven", Body: []byte("# Eleven\n")}
p.save()
p = &Page{Name: "testdata/pagination/index", Body: []byte(`# Index
* [one](one)
* [two](two)
@@ -144,3 +156,47 @@ func TestFeedPagination(t *testing.T) {
assert.Contains(t, body, `<atom:link href="https://example.org/view/testdata/pagination/index.rss?from=0&amp;n=3" rel="previous" type="application/rss+xml"/>`)
assert.Contains(t, body, `<atom:link href="https://example.org/view/testdata/pagination/index.rss?from=5&amp;n=3" rel="next" type="application/rss+xml"/>`)
}
func TestFeedYearArchives(t *testing.T) {
cleanup(t, "testdata/archives")
p := &Page{Name: "testdata/archives/index", Body: []byte(`# Archives
my bent fingers hurt
keyboard rattling in the dark
but no child in sight
`)}
p.save()
body := assert.HTTPBody(makeHandler(viewHandler, false, http.MethodGet), "GET",
"/view/testdata/archives/index.rss", nil)
year, err := p.BlogYear()
assert.Greater(t, year, 0)
assert.NoError(t, err)
prevLink := fmt.Sprintf(`<atom:link href="https://example.org/view/testdata/archives/%d.rss?n=10" rel="previous" type="application/rss+xml"/>`, year+1)
nextLink := fmt.Sprintf(`<atom:link href="https://example.org/view/testdata/archives/%d.rss?n=10" rel="next" type="application/rss+xml"/>`, year-1)
assert.NotContains(t, body, prevLink)
assert.NotContains(t, body, nextLink)
p = &Page{Name: fmt.Sprintf("testdata/archives/%d", year-1), Body: []byte(`# Previously
I have seen it all
invasion and denial
and cold winter hearts
`)}
p.save()
body = assert.HTTPBody(makeHandler(viewHandler, false, http.MethodGet), "GET",
"/view/testdata/archives/index.rss", nil)
assert.NotContains(t, body, prevLink)
assert.Contains(t, body, nextLink)
p = &Page{Name: fmt.Sprintf("testdata/archives/%d", year+1), Body: []byte(`# Coming
A night of thunder
lightning, children, it's the war
of our New Year's Eve
`)}
p.save()
body = assert.HTTPBody(makeHandler(viewHandler, false, http.MethodGet), "GET",
"/view/testdata/archives/index.rss", nil)
assert.Contains(t, body, prevLink)
assert.Contains(t, body, nextLink)
}

View File

@@ -4,17 +4,18 @@ import (
"context"
"flag"
"fmt"
"io"
"os"
"regexp"
"sort"
"strings"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/google/subcommands"
"github.com/hexops/gotextdiff"
"github.com/hexops/gotextdiff/myers"
"github.com/hexops/gotextdiff/span"
"io"
"os"
"regexp"
"sort"
"strings"
)
type hashtagsCmd struct {
@@ -166,8 +167,7 @@ func hashtagName(namesMap map[string]string, hashtag string, docids []docid) str
doc := markdown.Parse(p.Body, parser)
ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
if entering {
switch v := node.(type) {
case *ast.Link:
if v, ok := node.(*ast.Link); ok {
for _, attr := range v.AdditionalAttributes {
if attr == `class="tag"` {
tagName := []byte("")
@@ -181,7 +181,7 @@ func hashtagName(namesMap map[string]string, hashtag string, docids []docid) str
if strings.EqualFold(hashtag, strings.ReplaceAll(tag, " ", "_")) {
_, ok := candidate[tag]
if ok {
candidate[tag] += 1
candidate[tag]++
} else {
candidate[tag] = 1
}

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestHashtagsCmd(t *testing.T) {

View File

@@ -4,11 +4,12 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"html/template"
"io"
"os"
"strings"
"github.com/google/subcommands"
)
type htmlCmd struct {
@@ -41,7 +42,7 @@ func htmlCli(w io.Writer, template string, args []string) subcommands.ExitStatus
return subcommands.ExitFailure
}
p := &Page{Name: "stdin", Body: body}
return p.printHtml(w, template)
return p.printHTML(w, template)
}
for _, name := range args {
if !strings.HasSuffix(name, ".md") {
@@ -54,7 +55,7 @@ func htmlCli(w io.Writer, template string, args []string) subcommands.ExitStatus
fmt.Fprintf(os.Stderr, "Cannot load %s: %s\n", name, err)
return subcommands.ExitFailure
}
status := p.printHtml(w, template)
status := p.printHTML(w, template)
if status != subcommands.ExitSuccess {
return status
}
@@ -62,11 +63,11 @@ func htmlCli(w io.Writer, template string, args []string) subcommands.ExitStatus
return subcommands.ExitSuccess
}
func (p *Page) printHtml(w io.Writer, fn string) subcommands.ExitStatus {
func (p *Page) printHTML(w io.Writer, fn string) subcommands.ExitStatus {
if fn == "" {
// do not handle title
p.renderHtml()
_, err := fmt.Fprintln(w, p.Html)
p.renderHTML()
_, err := fmt.Fprintln(w, p.HTML)
if err != nil {
fmt.Fprintf(os.Stderr, "Cannot write to stdout: %s\n", err)
return subcommands.ExitFailure
@@ -74,7 +75,7 @@ func (p *Page) printHtml(w io.Writer, fn string) subcommands.ExitStatus {
return subcommands.ExitSuccess
}
p.handleTitle(true)
p.renderHtml()
p.renderHTML()
t, err := template.ParseFiles(fn)
if err != nil {
fmt.Fprintf(os.Stderr, "Cannot parse template %s for %s: %s\n", fn, p.Name, err)

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestHtmlCmd(t *testing.T) {

View File

@@ -5,7 +5,6 @@
package main
import (
"golang.org/x/exp/constraints"
"html/template"
"io/fs"
"log"
@@ -13,6 +12,8 @@ import (
"sort"
"strings"
"sync"
"golang.org/x/exp/constraints"
)
type docid uint
@@ -23,15 +24,15 @@ type docid uint
// It depends on the fact that Title is always plain text.
type ImageData struct {
Title, Name string
Html template.HTML
HTML template.HTML
}
// indexStore controls access to the maps used for search. Make sure to lock and unlock as appropriate.
type indexStore struct {
sync.RWMutex
// next_id is the number of the next document added to the index
next_id docid
// nextID is the number of the next document added to the index
nextID docid
// index is an inverted index mapping tokens to document ids.
token map[string][]docid
@@ -54,7 +55,7 @@ func init() {
// reset the index. This assumes that the index is locked. It's useful for tests.
func (idx *indexStore) reset() {
idx.next_id = 0
idx.nextID = 0
idx.token = make(map[string][]docid)
idx.documents = make(map[docid]string)
idx.titles = make(map[string]string)
@@ -64,8 +65,8 @@ func (idx *indexStore) reset() {
// addDocument adds the text as a new document. This assumes that the index is locked!
// The hashtags (only!) are used as tokens. They are stored in lower case.
func (idx *indexStore) addDocument(text []byte) docid {
id := idx.next_id
idx.next_id++
id := idx.nextID
idx.nextID++
for _, token := range hashtags(text) {
token = strings.ToLower(token)
ids := idx.token[token]
@@ -147,9 +148,8 @@ func (idx *indexStore) walk(fp string, info fs.FileInfo, err error) error {
if fp != "." && strings.HasPrefix(filepath.Base(fp), ".") {
if info.IsDir() {
return filepath.SkipDir
} else {
return nil
}
return nil
}
// skipp all but page files
if !strings.HasSuffix(fp, ".md") {
@@ -238,11 +238,12 @@ func intersection[T constraints.Ordered](a []T, b []T) []T {
r := make([]T, 0, maxLen)
var i, j int
for i < len(a) && j < len(b) {
if a[i] < b[j] {
switch {
case a[i] < b[j]:
i++
} else if a[i] > b[j] {
case a[i] > b[j]:
j++
} else {
default:
r = append(r, a[i])
i++
j++

View File

@@ -1,9 +1,10 @@
package main
import (
"github.com/stretchr/testify/assert"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestIndexAdd(t *testing.T) {

View File

@@ -2,9 +2,10 @@ package main
import (
"errors"
"github.com/pemistahl/lingua-go"
"os"
"strings"
"github.com/pemistahl/lingua-go"
)
// getLanguages returns the environment variable ODDMU_LANGUAGES or all languages.

View File

@@ -1,9 +1,10 @@
package main
import (
"github.com/stretchr/testify/assert"
"os"
"testing"
"github.com/stretchr/testify/assert"
)
func TestAllLanguage(t *testing.T) {

View File

@@ -4,10 +4,11 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"io"
"os"
"strings"
"github.com/google/subcommands"
)
type linksCmd struct {

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestLinksCmd(t *testing.T) {

View File

@@ -4,11 +4,12 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"io"
"os"
"path/filepath"
"strings"
"github.com/google/subcommands"
)
type listCmd struct {

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestListCmd(t *testing.T) {

View File

@@ -40,7 +40,7 @@ ServerAdmin alex@alexschroeder.ch
<VirtualHost *:443>
ServerName transjovian.org
SSLEngine on
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*)|sitemap\.xml)?$" \
"http://localhost:8080/$1"
</VirtualHost>
```
@@ -106,13 +106,13 @@ ServerAdmin alex@alexschroeder.ch
ServerName transjovian.org
ProxyPassMatch "^/((view|diff|search|archive)/(.*))?$" \
"http://localhost:8080/$1"
RedirectMatch "^/((edit|save|add|append|upload|drop)/(.*))?$" \
RedirectMatch "^/((edit|save|add|append|upload|drop)/(.*)|sitemap\.xml)?$" \
"https://transjovian.org/$1"
</VirtualHost>
<VirtualHost *:443>
ServerName transjovian.org
SSLEngine on
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*)|sitemap\.xml)?$" \
"http://localhost:8080/$1"
</VirtualHost>
```
@@ -144,7 +144,7 @@ You probably want to serve some static files as well (see *Serve static files*).
In that case, you need to use the ProxyPassMatch directive.
```
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*))?$" \
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*)|sitemap\.xml)?$" \
"unix:/run/oddmu/oddmu.sock|http://localhost/$1"
```
@@ -159,7 +159,7 @@ A workaround is to add the redirect manually and drop the question-mark:
```
RedirectMatch "^/$" "/view/index"
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*))$" \
ProxyPassMatch "^/((view|preview|diff|edit|save|add|append|upload|drop|search|archive)/(.*)|sitemap\.xml)$" \
"unix:/run/oddmu/oddmu.sock|http://localhost/$1"
```
@@ -213,8 +213,9 @@ The way Oddmu handles subdirectories is that all files and directories are
visible, except for "hidden" files and directories (whose name starts with a
period). Specifically, do not rely on Apache to hide locations in subdirectories
from public view. Search reveals the existence of these pages and produces an
extract, even if users cannot follow the links. Archive links pack all the
subdirectories, including locations you may have hidden from view using Apache.
extract, even if users cannot follow the links. The Sitemap lists all pages,
including subdirectories. Archive links pack all the subdirectories, including
locations you may have hidden from view using Apache.
If you to treat subdirectories as separate sites, you need to set the
environment variable ODDMU_FILTER to a regular expression matching the those

View File

@@ -5,7 +5,7 @@
.nh
.ad l
.\" Begin generated content:
.TH "ODDMU-FEED" "1" "2025-08-31"
.TH "ODDMU-FEED" "1" "2025-12-31"
.PP
.SH NAME
.PP
@@ -26,6 +26,12 @@ Unlike the feeds generated by the \fBstatic\fR subcommand, the \fBfeed\fR comman
not limit the feed to the ten most recent items.\& Instead, all items on the list
are turned into feed items.\&
.PP
Furthermore, if the items on the list are blog posts (their page name starts
with an ISO date), then this ISO date is used for the last update date to the
page instead of the last modification time of the file.\& The idea, more or less,
is that this feed is an archive feed and that in this context the creation date
is more important than the last modification date.\&
.PP
.SH EXAMPLES
.PP
Generate "emacs.\&rss" from "emacs.\&md":

View File

@@ -19,6 +19,12 @@ Unlike the feeds generated by the *static* subcommand, the *feed* command does
not limit the feed to the ten most recent items. Instead, all items on the list
are turned into feed items.
Furthermore, if the items on the list are blog posts (their page name starts
with an ISO date), then this ISO date is used for the last update date to the
page instead of the last modification time of the file. The idea, more or less,
is that this feed is an archive feed and that in this context the creation date
is more important than the last modification date.
# EXAMPLES
Generate "emacs.rss" from "emacs.md":

View File

@@ -6,13 +6,13 @@ oddmu-filter - keeping subdirectories separate
# DESCRIPTION
There are actions such as searching and archiving that act on multiple pages,
not just a single page. These actions walk the directory tree, including all
subdirectories. In some cases, this is not desirable.
There are actions such as producing the sitemap, searching and archiving that
act on multiple pages, not just a single page. These actions walk the directory
tree, including all subdirectories. In some cases, this is not desirable.
Sometimes, subdirectories are separate sites, like the sites of other projects
or different people. Depending on how you think about it, you might not want to
include those "sites" in searches or archives of the whole site.
include those "sites" in searches, sitemaps or archives of the whole site.
Since directory tree actions always start in the directory the visitor is
currently looking at, directory tree actions starting in a "separate site"

View File

@@ -19,7 +19,7 @@ The site is defined in "/etc/nginx/sites-available/default", in the _server_
section. Add a new _location_ section after the existing _location_ section:
```
location ~ ^/(view|preview|diff|edit|save|add|append|upload|drop|search|archive)/ {
location ~ ^/(view|preview|diff|edit|save|add|append|upload|drop|search|sitemap|archive)/ {
proxy_pass http://localhost:8080;
}
```
@@ -81,7 +81,7 @@ server configuration. On a Debian system, that'd be in
"/etc/nginx/sites-available/default".
```
location ~ ^/(view|preview|diff|edit|save|add|append|upload|drop|search|archive)/ {
location ~ ^/(view|preview|diff|edit|save|add|append|upload|drop|search|sitemap|archive)/ {
proxy_pass http://unix:/run/oddmu/oddmu.sock:;
}
```

View File

@@ -1,11 +1,11 @@
.\" Generated by scdoc 1.11.3
.\" Generated by scdoc 1.11.4
.\" Complete documentation for this program is not available as a GNU info page
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.nh
.ad l
.\" Begin generated content:
.TH "ODDMU-RELEASES" "7" "2025-09-28"
.TH "ODDMU-RELEASES" "7" "2026-01-01"
.PP
.SH NAME
.PP
@@ -15,6 +15,53 @@ oddmu-releases - what'\&s new?\&
.PP
This page lists user-visible features and template changes to consider.\&
.PP
.SS 1.20 (unreleased)
.PP
Add -shrink and -glob options to the \fIstatic\fR subcommand.\&
.PP
Some tools were used to check the code (goimports, golint, gocritic).\&
Unfortunately, the resulting changes necessitates a change in the templates
("feed.\&html", "preview.\&html", "search.\&html", "static.\&html", "view.\&html"):
"{{.\&Html}}" must be changed to "{{.\&HTML}}".\& One way to do this:
.PP
.nf
.RS 4
find \&. -regex \&'\&.*/(feed|preview|search|static|view).html\&'
-exec sed -i~ \&'s/{{\&.Html}}/{{\&.HTML}}/g\&' \&'{}\&' \&'+\&'
.fi
.RE
.PP
The \fIfeed\fR subcommand uses the page URL to extract a pubDate instead of relying
on the file'\&s last modified time.\& For a complete feed (an archive), the last
modified time is less important.\&
.PP
The feed for the index page is paginated, like other feeds.\& But since it grows
faster than any of the feeds for hashtag pages, presumably, an extra features
was added: on the first and on the last page of the feed, a link to the next or
the previous year is added, if such a page exists.\& This works if at beginning of
every year, you move all the entries on to a dedicated year page.\& You need to
add the necessary links to the feed template ("feed.\&html").\& See
\fIoddmu-templates\fR(5) for more.\&
.PP
Example:
.PP
.nf
.RS 4
<rss xmlns:atom="http://www\&.w3\&.org/2005/Atom" version="2\&.0"
xmlns:fh="http://purl\&.org/syndication/history/1\&.0">
{{if \&.PrevYear}}
<atom:link href="https://example\&.org/view/{{\&.Dir}}{{\&.PrevYear}}\&.rss?n={{\&.N}}"
rel="previous" type="application/rss+xml"/>
{{end}}
{{if \&.NextYear}}
<atom:link href="https://example\&.org/view/{{\&.Dir}}{{\&.NextYear}}\&.rss?n={{\&.N}}"
rel="next" type="application/rss+xml"/>
{{end}}
.fi
.RE
.PP
.SS 1.19 (2025)
.PP
Add \fIfeed\fR subcommand.\& This produces a "complete" feed.\&

View File

@@ -8,6 +8,49 @@ oddmu-releases - what's new?
This page lists user-visible features and template changes to consider.
## 1.20 (unreleased)
Add -shrink and -glob options to the _static_ subcommand.
Some tools were used to check the code (goimports, golint, gocritic).
Unfortunately, the resulting changes necessitates a change in the templates
("feed.html", "preview.html", "search.html", "static.html", "view.html"):
"{{.Html}}" must be changed to "{{.HTML}}". One way to do this:
```
find . -regex '.*/\(feed\|preview\|search\|static\|view\)\.html' \
-exec sed -i~ 's/{{.Html}}/{{.HTML}}/g' '{}' '+'
```
The _feed_ subcommand uses the page URL to extract a pubDate instead of relying
on the file's last modified time. For a complete feed (an archive), the last
modified time is less important.
The feed for the index page is paginated, like other feeds. But since it grows
faster than any of the feeds for hashtag pages, presumably, an extra features
was added: on the first and on the last page of the feed, a link to the next or
the previous year is added, if such a page exists. This works if at beginning of
every year, you move all the entries on to a dedicated year page. You need to
add the necessary links to the feed template ("feed.html"). See
_oddmu-templates_(5) for more.
Example:
```
<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"
xmlns:fh="http://purl.org/syndication/history/1.0">
{{if .PrevYear}}
<atom:link href="https://example.org/view/{{.Dir}}{{.PrevYear}}.rss?n={{.N}}"
rel="previous" type="application/rss+xml"/>
{{end}}
{{if .NextYear}}
<atom:link href="https://example.org/view/{{.Dir}}{{.NextYear}}.rss?n={{.N}}"
rel="next" type="application/rss+xml"/>
{{end}}
```
## 1.19 (2025)
Add _feed_ subcommand. This produces a "complete" feed.

38
man/oddmu-sitemap.1.txt Normal file
View File

@@ -0,0 +1,38 @@
ODDMU-SITEMAP(1)
# NAME
oddmu-sitemap - print static sitemap.xml
# SYNOPSIS
*oddmu sitemap* [-base URL]
# DESCRIPTION
The "sitemap" subcommand prints the list of all pages in Sitemap format. Oddmu
already serves the sitemap at the URL "/sitemap.xml" but if you'd prefer to
provide a static file, use this command and redirect the output to a file called
"sitemap.xml" in your document root at regular intervals.
If you do this, don't proxy the "/sitemap" URL in the web server configuration.
Your "robots.txt" file, if you have one, should point at the sitemap you
provide.
# OPTIONS
*-base* _URL_
The base URL is something like "https://example.org/view/".
*-filter* _regexp_
A regular expression matching the pages to exclude from the sitemap.
This emulates the effect of the ODDMU_FILTER environment variable.
# SEE ALSO
_oddmu_(1), _oddmu-filter_(7), _oddmu-apache_(1), _oddmu-nginx_(1),
https://www.sitemaps.org/
# AUTHORS
Maintained by Alex Schroeder <alex@gnu.org>.

View File

@@ -1,11 +1,11 @@
.\" Generated by scdoc 1.11.3
.\" Generated by scdoc 1.11.4
.\" Complete documentation for this program is not available as a GNU info page
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.nh
.ad l
.\" Begin generated content:
.TH "ODDMU-STATIC" "1" "2025-08-31"
.TH "ODDMU-STATIC" "1" "2025-12-05"
.PP
.SH NAME
.PP
@@ -13,7 +13,7 @@ oddmu-static - create a static copy of the site
.PP
.SH SYNOPSIS
.PP
\fBoddmu static\fR \fIdir-name\fR
\fBoddmu static\fR [\fB\fR-jobs\fB\fR \fIn\fR] [\fB\fR-glob\fB\fR \fIpattern\fR] [\fB\fR-shrink\fB\fR] \fIdir-name\fR
.PP
.SH DESCRIPTION
.PP
@@ -32,19 +32,18 @@ no feed items are found, no feed is written.\& The feed is limited to the ten mo
recent items.\&
.PP
Hidden files and directories (starting with a ".\&") and backup files (ending with
a "~") are skipped.\&
a "\(ti") are skipped.\&
.PP
All other files are \fIhard linked\fR.\& This is done to save space: on a typical blog
the images take a lot more space than the text.\& On my blog in 2023 I had 2.\&62
GiB of JPG files and 0.\&02 GiB of Markdown files.\& There is no point in copying
all those images, most of the time.\&
.PP
Note, however: Hard links cannot span filesystems.\& A hard link is just an extra
name for the same file.\& This is why the destination directory for the static
site has to be on same filesystem as the current directory, if it contains any
other files besides Markdown files.\&
As hard links cannot span filesystems, all other files are \fIcopied\fR if the
destination directory for the static site is not on same filesystem as the
current directory.\&
.PP
Furthermore, in-place editing changes the file for all names.\& Avoid editing the
Note that in-place editing changes the file for all names.\& Avoid editing the
hard-linked files (anything that'\&s not a HTML file) in the destination
directory, just to be on the safe side.\& Usually you should be fine, as an editor
moves the file that'\&s being edited to a backup file and creates a new file.\& But
@@ -52,6 +51,35 @@ then again, who knows.\& A SQLite file, for example, would change in-place, and
therefore making changes to it in the destination directory would change the
original, too.\&
.PP
.SH OPTIONS
.PP
\fB\fR-jobs\fB\fR \fIn\fR
.RS 4
By default, two jobs are used to process the files.\& If your machine has
more cores, you can increase the number of jobs.\&
.PP
.RE
\fB\fR-glob\fB\fR \fIpattern\fR
.RS 4
By default, all files are used for the static export.\& You can limit the
files used by providing a shell file name pattern.\& A "*" matches any
number of characters; a "?\&" matches exactly one character; "[a-z]"
matches a character listed, including ranges; "[\(haa-z]" matches a
character not listed, including ranges; "\e" a backslash escapes the
following character.\& You must use quotes around the pattern if you are
using a shell as the shell would otherwise expand the pattern, resulting
in the error "Exactly one target directory is required".\&
.PP
.PP
.RE
\fB\fR-shrink\fB\fR
.RS 4
By default, images are linked or copied.\& With this option, JPEG, PNG and
WebP files are scaled down if more than 800 pixels wide and the quality
is set to 10% for JPEG and WebP files.\& This is \fIvery bad quality\fR but
the result is that these image files are very small.\&
.PP
.RE
.SH EXAMPLES
.PP
Generate a static copy of the site, but only loading language detection for

View File

@@ -6,7 +6,7 @@ oddmu-static - create a static copy of the site
# SYNOPSIS
*oddmu static* _dir-name_
*oddmu static* [**-jobs** _n_] [**-glob** _pattern_] [**-shrink**] _dir-name_
# DESCRIPTION
@@ -32,12 +32,11 @@ the images take a lot more space than the text. On my blog in 2023 I had 2.62
GiB of JPG files and 0.02 GiB of Markdown files. There is no point in copying
all those images, most of the time.
Note, however: Hard links cannot span filesystems. A hard link is just an extra
name for the same file. This is why the destination directory for the static
site has to be on same filesystem as the current directory, if it contains any
other files besides Markdown files.
As hard links cannot span filesystems, all other files are _copied_ if the
destination directory for the static site is not on same filesystem as the
current directory.
Furthermore, in-place editing changes the file for all names. Avoid editing the
Note that in-place editing changes the file for all names. Avoid editing the
hard-linked files (anything that's not a HTML file) in the destination
directory, just to be on the safe side. Usually you should be fine, as an editor
moves the file that's being edited to a backup file and creates a new file. But
@@ -45,6 +44,29 @@ then again, who knows. A SQLite file, for example, would change in-place, and
therefore making changes to it in the destination directory would change the
original, too.
# OPTIONS
**-jobs** _n_
By default, two jobs are used to process the files. If your machine has
more cores, you can increase the number of jobs.
**-glob** _pattern_
By default, all files are used for the static export. You can limit the
files used by providing a shell file name pattern. A "\*" matches any
number of characters; a "?" matches exactly one character; "[a-z]"
matches a character listed, including ranges; "[^a-z]" matches a
character not listed, including ranges; "\\" a backslash escapes the
following character. You must use quotes around the pattern if you are
using a shell as the shell would otherwise expand the pattern, resulting
in the error "Exactly one target directory is required".
**-shrink**
By default, images are linked or copied. With this option, JPEG, PNG and
WebP files are scaled down if more than 800 pixels wide and the quality
is set to 10% for JPEG and WebP files. This is _very bad quality_ but
the result is that these image files are very small.
# EXAMPLES
Generate a static copy of the site, but only loading language detection for

View File

@@ -1,11 +1,11 @@
.\" Generated by scdoc 1.11.3
.\" Generated by scdoc 1.11.4
.\" Complete documentation for this program is not available as a GNU info page
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.nh
.ad l
.\" Begin generated content:
.TH "ODDMU-TEMPLATES" "5" "2025-09-24" "File Formats Manual"
.TH "ODDMU-TEMPLATES" "5" "2026-01-01" "File Formats Manual"
.PP
.SH NAME
.PP
@@ -152,6 +152,26 @@ is non-zero (in which case this is not the first page) before using \fI{{.\&Prev
\fI{{.\&Next}}\fR is the item number where the next feed starts, if there are any
items left.\& If there are none, it'\&s value is 0.\&
.PP
\fI{{.\&PrevYear}}\fR is the year for the previous yearly archive.\& This is added on
the index page or on year pages.\& Year pages are pages whose name is just a
number (presumably a year).\& The property is only set on the first page of the
feed, if the previous year page exists.\& The previous year is one higher than the
year currently shown (if on a year page) or the current year (if looking at the
index), since the feed goes backwards in time as new entries appear at the top.\&
When looking at the page "2024" the previous page is "2025".\& Strangely enough,
if the current year is 2026 but a page "2027" already exists, and the feed for
the index page is generated, then "2027" (in the future) is the previous page.\&
If the current year is 2026, the feed of the index page points to "2025" as the
next year, if it exists.\& When the feed for "2025" is generated, however, the
previous year is not set, assuming that the "2026" page does not yet exist and
it is strange to consider the index page "the previous year" of "2025" in 2026.\&
This might change in the future.\& If it isn'\&t set, it'\&s value is 0.\&
.PP
\fI{{.\&NextYear}}\fR is the year for the next yearly archive.\& See above for an
explanation.\& The next year is one lower than the year currently shown (if on a
year page) or the current year (if looking at the index).\& If it isn'\&t set, it'\&s
value is 0.\&
.PP
.SS List
.PP
The list contains a directory name and an array of files.\&

View File

@@ -18,9 +18,9 @@ placeholders.
- _diff.html_ uses a _page_
- _edit.html_ uses a _page_
- _feed.html_ uses a _feed_
- _list.html_ uses a _list_
- _preview.html_ uses a _page_
- _search.html_ uses a _search_
- _sitemap.html_ uses a _sitemap_
- _static.html_ uses a _page_
- _upload.html_ uses an _upload_
- _view.html_ uses a _page_
@@ -126,31 +126,25 @@ is non-zero (in which case this is not the first page) before using _{{.Prev}}_.
_{{.Next}}_ is the item number where the next feed starts, if there are any
items left. If there are none, it's value is 0.
## List
_{{.PrevYear}}_ is the year for the previous yearly archive. This is added on
the index page or on year pages. Year pages are pages whose name is just a
number (presumably a year). The property is only set on the first page of the
feed, if the previous year page exists. The previous year is one higher than the
year currently shown (if on a year page) or the current year (if looking at the
index), since the feed goes backwards in time as new entries appear at the top.
When looking at the page "2024" the previous page is "2025". Strangely enough,
if the current year is 2026 but a page "2027" already exists, and the feed for
the index page is generated, then "2027" (in the future) is the previous page.
If the current year is 2026, the feed of the index page points to "2025" as the
next year, if it exists. When the feed for "2025" is generated, however, the
previous year is not set, assuming that the "2026" page does not yet exist and
it is strange to consider the index page "the previous year" of "2025" in 2026.
This might change in the future. If it isn't set, it's value is 0.
The list contains a directory name and an array of files.
_{{.Dir}}_ is the directory name that is being listed, percent-encoded.
_{{.Files}}_ is the array of files. To refer to them, you need to use a _{{range
.Files}}_ … _{{end}}_ construct.
Each file has the following attributes:
_{{.Name}}_ is the filename. The ".md" suffix for Markdown files is part of the
name (unlike page names).
_{{.Path}}_ is the page name, percent-encoded.
_{{.Title}}_ is the page title, if the file in question is a Markdown file.
_{{.IsDir}}_ is a boolean used to indicate that this file is a directory.
_{{.IsUp}}_ is a boolean used to indicate the entry for the parent directory
(the first file in the array, unless the directory being listed is the top
directory). The filename of this file is "..".
_{{.Date}}_ is the last modification date of the file.
_{{.NextYear}}_ is the year for the next yearly archive. See above for an
explanation. The next year is one lower than the year currently shown (if on a
year page) or the current year (if looking at the index). If it isn't set, it's
value is 0.
## Search
@@ -190,6 +184,16 @@ _{{.Name}}_ is the file name for use in URLs.
_{{.Html}}_ the image alt-text with a bold tag used to highlight the first
search term that matched.
## Sitemap
The sitemap contains a list of URLs, each with its location:
_{{.URL}}_ is the list of URLs.
Each URL has the following attributes:
_{{.Loc}}_ with the actual page URL.
## Upload
_{{.Dir}}_ is the directory where the uploaded file ends up, based on the URL

View File

@@ -56,6 +56,7 @@ directory:
- _/upload/dir/name_ shows a form to upload a file
- _/drop/dir/name_ saves an upload
- _/search/dir/?q=term_ to search for a term
- _/sitemap.xml_ to list the links to all the pages
- _/archive/dir/name.zip_ to download a zip file of a directory
When calling the _save_ and _append_ action, the page name is taken from the URL
@@ -324,6 +325,7 @@ Oddmu running as a webserver:
- _oddmu-notify_(1), on updating index, changes and hashtag pages
- _oddmu-replace_(1), on how to search and replace text
- _oddmu-search_(1), on how to run a search
- _oddmu-sitemap_(1), on generating a static sitemap.xml
- _oddmu-static_(1), on generating a static site
- _oddmu-toc_(1), on how to list the table of contents (toc) a page
- _oddmu-version_(1), on how to get all the build information from the binary

View File

@@ -1,7 +1,6 @@
package main
import (
"github.com/stretchr/testify/assert"
"go/parser"
"go/token"
"io/fs"
@@ -12,6 +11,8 @@ import (
"sort"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
// Does oddmu(1) link to all the other man pages?

View File

@@ -4,14 +4,15 @@ import (
"context"
"flag"
"fmt"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/google/subcommands"
"io"
"net/url"
"os"
"path"
"strings"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/google/subcommands"
)
type missingCmd struct {
@@ -32,6 +33,12 @@ func (cmd *missingCmd) SetFlags(f *flag.FlagSet) {
}
func (cmd *missingCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
n, err := index.load()
if err != nil {
fmt.Fprintf(os.Stderr, "Index load: %s\n", err)
return subcommands.ExitFailure
}
fmt.Fprintf(os.Stderr, "Indexed %d pages\n", n)
return missingCli(os.Stdout, &index)
}
@@ -94,8 +101,7 @@ func (p *Page) links() []string {
doc := markdown.Parse(p.Body, parser)
ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
if entering {
switch v := node.(type) {
case *ast.Link:
if v, ok := node.(*ast.Link); ok {
link := string(v.Destination)
url, err := url.Parse(link)
if err != nil {

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestMissingCmd(t *testing.T) {

View File

@@ -4,10 +4,11 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"io"
"os"
"strings"
"github.com/google/subcommands"
)
type notifyCmd struct {
@@ -31,7 +32,6 @@ func (cmd *notifyCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface
}
func notifyCli(w io.Writer, args []string) subcommands.ExitStatus {
index.load()
for _, name := range args {
if !strings.HasSuffix(name, ".md") {
fmt.Fprintf(os.Stderr, "%s does not end in '.md'\n", name)

19
page.go
View File

@@ -2,7 +2,6 @@ package main
import (
"bytes"
"github.com/microcosm-cc/bluemonday"
"html/template"
"log"
"net/url"
@@ -12,6 +11,8 @@ import (
"regexp"
"strings"
"time"
"github.com/microcosm-cc/bluemonday"
)
// Page is a struct containing information about a single page. Title is the title extracted from the page content using
@@ -21,7 +22,7 @@ type Page struct {
Title string
Name string
Body []byte
Html template.HTML
HTML template.HTML
Hashtags []string
}
@@ -29,7 +30,7 @@ type Page struct {
// the Name "foo").
type Link struct {
Title string
Url string
URL string
}
// blogRe is a regular expression that matches blog pages. If the filename of a blog page starts with an ISO date
@@ -61,7 +62,7 @@ func nameEscape(s string) string {
}
// save saves a Page. The path is based on the Page.Name and gets the ".md" extension. Page.Body is saved, without any
// carriage return characters ("\r"). Page.Title and Page.Html are not saved. There is no caching. Before removing or
// carriage return characters ("\r"). Page.Title and Page.HTML are not saved. There is no caching. Before removing or
// writing a file, the old copy is renamed to a backup, appending "~". Errors are not logged but returned.
func (p *Page) save() error {
fp := filepath.FromSlash(p.Name) + ".md"
@@ -88,6 +89,8 @@ func (p *Page) save() error {
return os.WriteFile(fp, s, 0644)
}
// ModTime returns the last modification time of the page file. If the page does not exist, the current time is
// returned.
func (p *Page) ModTime() (time.Time, error) {
fp := filepath.FromSlash(p.Name) + ".md"
fi, err := os.Stat(fp)
@@ -120,7 +123,7 @@ func backup(fp string) error {
}
// loadPage loads a Page given a name. The path loaded is that Page.Name with the ".md" extension. The Page.Title is set
// to the Page.Name (and possibly changed, later). The Page.Body is set to the file content. The Page.Html remains
// to the Page.Name (and possibly changed, later). The Page.Body is set to the file content. The Page.HTML remains
// undefined (there is no caching).
func loadPage(name string) (*Page, error) {
name = strings.TrimPrefix(name, "./") // result of a path.TreeWalk starting with "."
@@ -145,10 +148,10 @@ func (p *Page) handleTitle(replace bool) {
}
}
// summarize sets Page.Html to an extract.
// summarize sets Page.HTML to an extract.
func (p *Page) summarize(q string) {
t := p.plainText()
p.Html = sanitizeStrict(snippets(q, t))
p.HTML = sanitizeStrict(snippets(q, t))
}
// IsBlog returns true if the page name starts with an ISO date
@@ -232,7 +235,7 @@ func (p *Page) Parents() []*Link {
if !ok {
title = "…"
}
link := &Link{Title: title, Url: strings.Repeat("../", len(elems)-i-1) + "index"}
link := &Link{Title: title, URL: strings.Repeat("../", len(elems)-i-1) + "index"}
links = append(links, link)
s += elems[i] + "/"
}

View File

@@ -1,9 +1,10 @@
package main
import (
"github.com/stretchr/testify/assert"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
)
func TestPageTitle(t *testing.T) {
@@ -58,12 +59,12 @@ And untouchable`)}
// "testdata/parents/children/something/index" is a sibling and doesn't count!
parents := p.Parents()
assert.Equal(t, "Welcome to Oddμ", parents[0].Title)
assert.Equal(t, "../../../../index", parents[0].Url)
assert.Equal(t, "../../../../index", parents[0].URL)
assert.Equal(t, "…", parents[1].Title)
assert.Equal(t, "../../../index", parents[1].Url)
assert.Equal(t, "../../../index", parents[1].URL)
assert.Equal(t, "Solar", parents[2].Title)
assert.Equal(t, "../../index", parents[2].Url)
assert.Equal(t, "../../index", parents[2].URL)
assert.Equal(t, "Lunar", parents[3].Title)
assert.Equal(t, "../index", parents[3].Url)
assert.Equal(t, "../index", parents[3].URL)
assert.Equal(t, 4, len(parents))
}

View File

@@ -2,12 +2,13 @@ package main
import (
"bytes"
"net/url"
"path"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/gomarkdown/markdown/html"
"github.com/gomarkdown/markdown/parser"
"net/url"
"path"
)
// wikiLink returns an inline parser function. This indirection is
@@ -89,12 +90,12 @@ func wikiRenderer() *html.Renderer {
return renderer
}
// renderHtml renders the Page.Body to HTML and sets Page.Html, Page.Hashtags, and escapes Page.Name.
func (p *Page) renderHtml() {
// renderHTML renders the Page.Body to HTML and sets Page.HTML, Page.Hashtags, and escapes Page.Name.
func (p *Page) renderHTML() {
parser, hashtags := wikiParser()
renderer := wikiRenderer()
maybeUnsafeHTML := markdown.ToHTML(p.Body, parser, renderer)
p.Html = unsafeBytes(maybeUnsafeHTML)
p.HTML = unsafeBytes(maybeUnsafeHTML)
p.Hashtags = *hashtags
}
@@ -133,8 +134,7 @@ func (p *Page) images() []ImageData {
doc := markdown.Parse(p.Body, parser)
ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
if entering {
switch v := node.(type) {
case *ast.Image:
if v, ok := node.(*ast.Image); ok {
// not an absolute URL, not a full URL, not a mailto: URI
text := toString(v)
if len(text) > 0 {
@@ -164,8 +164,7 @@ func toString(node ast.Node) string {
b := new(bytes.Buffer)
ast.WalkFunc(node, func(node ast.Node, entering bool) ast.WalkStatus {
if entering {
switch v := node.(type) {
case *ast.Text:
if v, ok := node.(*ast.Text); ok {
b.Write(v.Literal)
}
}

View File

@@ -1,8 +1,9 @@
package main
import (
"github.com/stretchr/testify/assert"
"testing"
"github.com/stretchr/testify/assert"
)
func TestPagePlainText(t *testing.T) {
@@ -19,14 +20,14 @@ func TestPageHtml(t *testing.T) {
Silver leaves shine bright
They droop, boneless, weak and sad
A cruel sun stares down`)}
p.renderHtml()
p.renderHTML()
r := `<h1 id="sun">Sun</h1>
<p>Silver leaves shine bright
They droop, boneless, weak and sad
A cruel sun stares down</p>
`
assert.Equal(t, r, string(p.Html))
assert.Equal(t, r, string(p.HTML))
}
func TestPageHtmlHashtag(t *testing.T) {
@@ -36,7 +37,7 @@ Too faint to focus, so far
I am cold, alone
#Haiku #Cold_Poets`)}
p.renderHtml()
p.renderHTML()
r := `<h1 id="comet">Comet</h1>
<p>Stars flicker above
@@ -45,7 +46,7 @@ I am cold, alone</p>
<p><a class="tag" href="/search/?q=%23Haiku">#Haiku</a> <a class="tag" href="/search/?q=%23Cold_Poets">#Cold Poets</a></p>
`
assert.Equal(t, r, string(p.Html))
assert.Equal(t, r, string(p.HTML))
}
func TestPageHtmlHashtagCornerCases(t *testing.T) {
@@ -53,13 +54,13 @@ func TestPageHtmlHashtagCornerCases(t *testing.T) {
ok # #o #ok
[oh #ok \#nok](ok)`)}
p.renderHtml()
p.renderHTML()
r := `<p>#</p>
<p>ok # <a class="tag" href="/search/?q=%23o">#o</a> <a class="tag" href="/search/?q=%23ok">#ok</a>
<a href="ok">oh #ok #nok</a></p>
`
assert.Equal(t, r, string(p.Html))
assert.Equal(t, r, string(p.HTML))
}
func TestPageHtmlWikiLink(t *testing.T) {
@@ -67,14 +68,14 @@ func TestPageHtmlWikiLink(t *testing.T) {
Blue and green and black
Sky and grass and [ragged cliffs](cliffs)
Our [[time together]]`)}
p.renderHtml()
p.renderHTML()
r := `<h1 id="photos-and-books">Photos and Books</h1>
<p>Blue and green and black
Sky and grass and <a href="cliffs">ragged cliffs</a>
Our <a href="time%20together">time together</a></p>
`
assert.Equal(t, r, string(p.Html))
assert.Equal(t, r, string(p.HTML))
}
func TestPageHtmlDollar(t *testing.T) {
@@ -82,34 +83,34 @@ func TestPageHtmlDollar(t *testing.T) {
Dragonfly hovers
darts chases turns lands and rests
A mighty jewel`)}
p.renderHtml()
p.renderHTML()
r := `<h1 id="no-dollar-can-buy-this">No $dollar$ can buy this</h1>
<p>Dragonfly hovers
darts chases turns lands and rests
A mighty jewel</p>
`
assert.Equal(t, r, string(p.Html))
assert.Equal(t, r, string(p.HTML))
}
func TestLazyLoadImages(t *testing.T) {
p := &Page{Body: []byte(`![](test.jpg)`)}
p.renderHtml()
assert.Contains(t, string(p.Html), "lazy")
p.renderHTML()
assert.Contains(t, string(p.HTML), "lazy")
}
// The fractions available in Latin 1 (?) are rendered.
func TestFractions(t *testing.T) {
p := &Page{Body: []byte(`1/4`)}
p.renderHtml()
assert.Contains(t, string(p.Html), "&frac14;")
p.renderHTML()
assert.Contains(t, string(p.HTML), "&frac14;")
}
// Other fractions are not rendered.
func TestNoFractions(t *testing.T) {
p := &Page{Body: []byte(`1/6`)}
p.renderHtml()
assert.Contains(t, string(p.Html), "1/6")
p.renderHTML()
assert.Contains(t, string(p.HTML), "1/6")
}
// webfinger
@@ -123,13 +124,13 @@ func TestAt(t *testing.T) {
accounts.Unlock()
// test account
p := &Page{Body: []byte(`My fedi handle is @alex@alexschroeder.ch.`)}
p.renderHtml()
assert.Contains(t, string(p.Html),
p.renderHTML()
assert.Contains(t, string(p.HTML),
`My fedi handle is <a class="account" href="https://social.alexschroeder.ch/@alex" title="@alex@alexschroeder.ch">@alex</a>.`)
// test escaped account
p = &Page{Body: []byte(`My fedi handle is \@alex@alexschroeder.ch. \`)}
p.renderHtml()
assert.Contains(t, string(p.Html),
p.renderHTML()
assert.Contains(t, string(p.HTML),
`My fedi handle is @alex@alexschroeder.ch.`)
// disable webfinger
useWebfinger = false

View File

@@ -20,6 +20,6 @@ func previewHandler(w http.ResponseWriter, r *http.Request, path string) {
body := strings.ReplaceAll(r.FormValue("body"), "\r", "")
p := &Page{Name: path, Body: []byte(body)}
p.handleTitle(true)
p.renderHtml()
p.renderHTML()
renderTemplate(w, p.Dir(), "preview", p)
}

View File

@@ -23,7 +23,7 @@ img { max-width: 100% }
</header>
<main>
<h1>Previewing {{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<hr>
<section id="edit">

View File

@@ -1,10 +1,11 @@
package main
import (
"github.com/stretchr/testify/assert"
"net/http"
"net/url"
"testing"
"github.com/stretchr/testify/assert"
)
func TestPreview(t *testing.T) {

View File

@@ -4,10 +4,6 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"github.com/hexops/gotextdiff"
"github.com/hexops/gotextdiff/myers"
"github.com/hexops/gotextdiff/span"
"io"
"io/fs"
"os"
@@ -15,6 +11,11 @@ import (
"regexp"
"slices"
"strings"
"github.com/google/subcommands"
"github.com/hexops/gotextdiff"
"github.com/hexops/gotextdiff/myers"
"github.com/hexops/gotextdiff/span"
)
type replaceCmd struct {
@@ -63,9 +64,8 @@ func replaceCli(w io.Writer, isConfirmed bool, isRegexp bool, args []string) sub
if fp != "." && strings.HasPrefix(filepath.Base(fp), ".") {
if info.IsDir() {
return filepath.SkipDir
} else {
return nil
}
return nil
}
// skipp all but page files
if !strings.HasSuffix(fp, ".md") {

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestReplaceCmd(t *testing.T) {

View File

@@ -67,11 +67,12 @@ func sortNames(tokens []string) func(a, b string) int {
na := unicode.IsNumber(ra)
rb, _ := utf8.DecodeRuneInString(b)
nb := unicode.IsNumber(rb)
if na && !nb {
switch {
case na && !nb:
return -1
} else if !na && nb {
case !na && nb:
return 1
} else if na && nb {
case na && nb:
if a < b {
return 1
} else if a > b {
@@ -99,7 +100,7 @@ func sortNames(tokens []string) func(a, b string) int {
// results.
const itemsPerPage = 20
// search returns a sorted []Page where each page contains an extract of the actual Page.Body in its Page.Html. Page
// search returns a sorted []Page where each page contains an extract of the actual Page.Body in its Page.HTML. Page
// size is 20. Specify either the page number to return, or that all the results should be returned. Only ask for all
// results if runtime is not an issue, like on the command line. The boolean return value indicates whether there are
// more results.
@@ -140,7 +141,7 @@ func search(q, dir, filter string, page int, all bool) ([]*Result, bool) {
if strings.Contains(title, term) {
re, err := re(term)
if err == nil {
img.Html = template.HTML(highlight(re, img.Title))
img.HTML = template.HTML(highlight(re, img.Title))
}
res = append(res, img)
continue ImageLoop
@@ -199,14 +200,15 @@ func filterNames(names, predicates []string) []string {
defer index.RUnlock()
for _, predicate := range predicates {
r := make([]string, 0)
if strings.HasPrefix(predicate, "title:") {
switch {
case strings.HasPrefix(predicate, "title:"):
token := predicate[6:]
for _, name := range names {
if strings.Contains(strings.ToLower(index.titles[name]), token) {
r = append(r, name)
}
}
} else if predicate == "blog:true" || predicate == "blog:false" {
case predicate == "blog:true" || predicate == "blog:false":
blog := predicate == "blog:true"
re := regexp.MustCompile(`(^|/)\d\d\d\d-\d\d-\d\d`)
for _, name := range names {
@@ -215,7 +217,7 @@ func filterNames(names, predicates []string) []string {
r = append(r, name)
}
}
} else {
default:
log.Printf("Unsupported predicate: %s", predicate)
}
names = intersection(names, r)

View File

@@ -40,9 +40,9 @@ button { background-color: #eee; color: inherit; border-radius: 4px; border-widt
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
{{range .Images}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.Html}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.HTML}}
{{end}}
</article>
{{end}}

View File

@@ -4,14 +4,15 @@ import (
"context"
"flag"
"fmt"
"github.com/google/subcommands"
"github.com/muesli/reflow/wordwrap"
"io"
"net/url"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/google/subcommands"
"github.com/muesli/reflow/wordwrap"
)
type searchCmd struct {
@@ -70,14 +71,15 @@ func searchCli(w io.Writer, cmd *searchCmd, args []string) subcommands.ExitStatu
fmt.Fprint(os.Stderr, " results\n")
}
}
if cmd.extract {
switch {
case cmd.extract:
searchExtract(w, items)
} else if cmd.files {
case cmd.files:
for _, p := range items {
name := filepath.FromSlash(p.Name) + ".md\n"
fmt.Fprintf(w, name)
}
} else {
default:
for _, p := range items {
name := p.Name
if strings.HasPrefix(name, dir) {
@@ -98,7 +100,7 @@ func searchExtract(w io.Writer, items []*Result) {
match := func(s string) string { return "\x1b[1m" + s + "\x1b[0m" } // bold
re := regexp.MustCompile(`<b>(.*?)</b>`)
for _, p := range items {
s := re.ReplaceAllString(string(p.Html), match(`$1`))
s := re.ReplaceAllString(string(p.HTML), match(`$1`))
fmt.Fprintln(w, heading(p.Title))
if p.Name != p.Title {
fmt.Fprintln(w, p.Name)

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
func TestSearchCmd(t *testing.T) {

View File

@@ -2,11 +2,12 @@ package main
import (
"fmt"
"github.com/stretchr/testify/assert"
"net/http"
"net/url"
"slices"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSortNames(t *testing.T) {
@@ -262,7 +263,7 @@ Please call me, my love.
assert.NotEmpty(t, items[0].Images)
assert.Equal(t, "phone call", items[0].Images[0].Title)
assert.Equal(t, "phone <b>call</b>", string(items[0].Images[0].Html))
assert.Equal(t, "phone <b>call</b>", string(items[0].Images[0].HTML))
assert.Equal(t, "testdata/images/2024-07-21.jpg", items[0].Images[0].Name)
assert.Empty(t, items[1].Images)

48
sitemap.go Normal file
View File

@@ -0,0 +1,48 @@
package main
import (
"log"
"net/http"
"os"
"regexp"
)
type SitemapURL struct {
Loc string
}
type Sitemap struct {
URL []*SitemapURL
}
// sitemapHandler lists all the pages. See https://www.sitemaps.org/protocol.html for more. It takes the
// ODDMU_FILTER environment variable into account.
func sitemapHandler(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/sitemap.xml" {
http.NotFound(w, r)
} else {
w.Write([]byte(`<?xml version="1.0" encoding="UTF-8"?>`))
base := r.URL.Scheme + "://" + r.URL.Host + "/view/"
filter := os.Getenv("ODDMU_FILTER")
renderTemplate(w, ".", "sitemap", sitemap(&index, base, filter))
}
}
// sitemap generates the list of URLs. A reference to the index needs to be provided to make it easier to write
// tests. Exclude pages matching the filter.
func sitemap(idx *indexStore, base, filter string) Sitemap {
url := make([]*SitemapURL, 0)
re, err := regexp.Compile(filter)
if err != nil {
log.Println("ODDMU_FILTER does not compile:", filter, err)
return Sitemap{URL: url}
}
idx.RLock()
defer idx.RUnlock()
for name := range idx.titles {
if !re.MatchString(name) {
url = append(url, &SitemapURL{Loc: base + name})
}
}
return Sitemap{URL: url}
}

3
sitemap.html Normal file
View File

@@ -0,0 +1,3 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
{{range .URL}}<url><loc>{{.Loc}}</loc></url>
{{end}}</urlset>

62
sitemap_cmd.go Normal file
View File

@@ -0,0 +1,62 @@
package main
import (
"context"
"fmt"
"flag"
"io"
"log"
"os"
"github.com/google/subcommands"
)
type sitemapCmd struct {
base string
filter string
}
func (cmd *sitemapCmd) SetFlags(f *flag.FlagSet) {
f.StringVar(&cmd.base, "base", "http://localhost:8080/view/", "the base URL for the sitemap")
f.StringVar(&cmd.filter, "filter", "", "a regular expression to filter pages")
}
func (*sitemapCmd) Name() string { return "sitemap" }
func (*sitemapCmd) Synopsis() string { return "list all the pages known in Sitemap format" }
func (*sitemapCmd) Usage() string {
return `sitemap [-base URL] [-filter regex]:
Print all the pages known in Sitemap format.
See https://www.sitemaps.org/ for more.
`
}
func (cmd *sitemapCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
n, err := index.load()
if err != nil {
fmt.Fprintf(os.Stderr, "Index load: %s\n", err)
return subcommands.ExitFailure
}
fmt.Fprintf(os.Stderr, "Indexed %d pages\n", n)
return sitemapCli(os.Stdout, &index, cmd.base, cmd.filter)
}
// sitemapCli implements the printing of a Sitemap. In order to make testing easier, it takes a Writer and an
// indexStore. The Writer is important so that test code can provide a buffer instead of os.Stdout; the indexStore
// is important so that test code can ensure no other test running in parallel can interfere with the list of known
// pages (by adding or deleting pages).
func sitemapCli(w io.Writer, idx *indexStore, base, filter string) subcommands.ExitStatus {
loadTemplates()
template := "sitemap.html"
t := templates.template[template]
if t == nil {
log.Println("Template not found:", template)
return subcommands.ExitFailure
}
w.Write([]byte(`<?xml version="1.0" encoding="UTF-8"?>` + "\n"))
err := t.Execute(w, sitemap(idx, base, filter))
if err != nil {
log.Println(err)
return subcommands.ExitFailure
}
return subcommands.ExitSuccess
}

18
sitemap_cmd_test.go Normal file
View File

@@ -0,0 +1,18 @@
package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
)
func TestSitemapCmd(t *testing.T) {
b := new(bytes.Buffer)
s := sitemapCli(b, minimalIndex(t), "https://example.org/view/", "^themes/")
assert.Equal(t, subcommands.ExitSuccess, s)
assert.Contains(t, b.String(), "https://example.org/view/index")
assert.Contains(t, b.String(), "https://example.org/view/README")
assert.NotContains(t, b.String(), "https://example.org/view/themes/")
}

View File

@@ -90,9 +90,9 @@ func snippets(q string, s string) string {
}
}
t = s[start:end]
res = res + t
res += t
if len(s) > end {
res = res + " …"
res += " …"
}
// truncate text to avoid rematching the same string.
s = s[end:]

View File

@@ -1,8 +1,9 @@
package main
import (
"github.com/stretchr/testify/assert"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSnippets(t *testing.T) {

View File

@@ -19,7 +19,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -5,10 +5,8 @@ import (
"context"
"flag"
"fmt"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/gomarkdown/markdown/html"
"github.com/google/subcommands"
"image/jpeg"
"io"
"io/fs"
"net/url"
"os"
@@ -16,14 +14,28 @@ import (
"slices"
"strings"
"time"
"github.com/disintegration/imaging"
"github.com/edwvee/exiffix"
"github.com/gen2brain/webp"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/gomarkdown/markdown/html"
"github.com/google/subcommands"
)
type staticCmd struct {
jobs int
jobs int
shrink bool
glob string
verbose bool
}
func (cmd *staticCmd) SetFlags(f *flag.FlagSet) {
f.IntVar(&cmd.jobs, "jobs", 2, "how many jobs to use")
f.BoolVar(&cmd.shrink, "shrink", false, "shrink images by decreasing the quality")
f.StringVar(&cmd.glob, "glob", "", "only export files matching this shell file name pattern")
f.BoolVar(&cmd.verbose, "verbose", false, "print the files as they are being processed")
}
func (*staticCmd) Name() string { return "static" }
@@ -38,11 +50,11 @@ func (*staticCmd) Usage() string {
func (cmd *staticCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
args := f.Args()
if len(args) != 1 {
fmt.Fprintln(os.Stderr, "Exactly one target directory is required")
fmt.Fprintln(os.Stderr, "Exactly one target directory is require", args)
return subcommands.ExitFailure
}
dir := filepath.Clean(args[0])
return staticCli(".", dir, cmd.jobs, false)
return staticCli(".", dir, cmd.jobs, cmd.glob, cmd.shrink, cmd.verbose, false)
}
type args struct {
@@ -52,20 +64,20 @@ type args struct {
// staticCli generates a static site in the designated directory. The quiet flag is used to suppress output when running
// tests. The source directory cannot be set from the command-line. The current directory (".") is assumed.
func staticCli(source, target string, jobs int, quiet bool) subcommands.ExitStatus {
func staticCli(source, target string, jobs int, glob string, shrink, verbose, quiet bool) subcommands.ExitStatus {
index.load()
index.RLock()
defer index.RUnlock()
loadLanguages()
loadTemplates()
tasks := make(chan args)
results := make(chan error)
done := make(chan bool)
tasks := make(chan args, 10000)
results := make(chan error, jobs)
done := make(chan bool, jobs)
stop := make(chan error)
for i := 0; i < jobs; i++ {
go staticWorker(tasks, results, done)
go staticWorker(tasks, results, done, shrink, verbose)
}
go staticWalk(source, target, tasks, stop)
go staticWalk(source, target, glob, tasks, stop)
go staticWatch(jobs, results, done)
n, err := staticProgressIndicator(results, stop, quiet)
if !quiet {
@@ -81,13 +93,15 @@ func staticCli(source, target string, jobs int, quiet bool) subcommands.ExitStat
// staticWalk walks the source directory tree. Any directory it finds, it recreates in the target directory. Any file it
// finds, it puts into the tasks channel for the staticWorker. When the directory walk is finished, the tasks channel is
// closed. If there's an error on the stop channel, the walk returns that error.
func staticWalk(source, target string, tasks chan (args), stop chan (error)) {
func staticWalk(source, target, glob string, tasks chan (args), stop chan (error)) {
// The error returned here is what's in the stop channel but at the very end, a worker might return an error
// even though the walk is already done. This is why we cannot rely on the return value of the walk.
filepath.Walk(source, func(fp string, info fs.FileInfo, err error) error {
n := 0
err := filepath.Walk(source, func(fp string, info fs.FileInfo, err error) error {
if err != nil {
return err
}
// don't wait for the stop channel
select {
case err := <-stop:
return err
@@ -96,14 +110,30 @@ func staticWalk(source, target string, tasks chan (args), stop chan (error)) {
if fp != "." && strings.HasPrefix(filepath.Base(fp), ".") {
if info.IsDir() {
return filepath.SkipDir
} else {
return nil
}
return nil
}
// skip backup files, avoid recursion
if strings.HasSuffix(fp, "~") || strings.HasPrefix(fp, target) {
return nil
}
// skip templates
if slices.Contains(templateFiles, filepath.Base(fp)) {
return nil
}
// skip files that don't match the glob, if set
if fp != "." && glob != "" {
match, err := filepath.Match(glob, fp)
if err != nil {
return err // abort
}
if !match {
if info.IsDir() {
return filepath.SkipDir
}
return nil
}
}
// determine the actual target: if source is a/ and target is b/ and path is a/file, then the
// target is b/file
var actualTarget string
@@ -115,18 +145,32 @@ func staticWalk(source, target string, tasks chan (args), stop chan (error)) {
}
actualTarget = filepath.Join(target, fp[len(source):])
}
// recreate subdirectories
// recreate subdirectories, ignore existing ones
if info.IsDir() {
return os.Mkdir(actualTarget, 0755)
os.Mkdir(actualTarget, 0755)
return nil
}
// Markdown files end up as HTML files
if strings.HasSuffix(actualTarget, ".md") {
actualTarget = actualTarget[:len(actualTarget)-3] + ".html"
}
// do the task if the target file doesn't exist or if the source file is newer
other, err := os.Stat(actualTarget)
if err != nil || info.ModTime().After(other.ModTime()) {
if err == nil {
fmt.Println(fp, info.ModTime(), other.ModTime(), info.ModTime().After(other.ModTime()))
}
n++
tasks <- args{source: fp, target: actualTarget, info: info}
}
return nil
}
})
if err != nil {
fmt.Println(err)
} else {
fmt.Printf("\r%d files to process\n", n)
}
close(tasks)
}
@@ -141,55 +185,114 @@ func staticWatch(jobs int, results chan (error), done chan (bool)) {
// staticWorker takes arguments off the tasks channel (the file to process) and put results in the results channel (any
// errors encountered); when they're done they send true on the done channel.
func staticWorker(tasks chan (args), results chan (error), done chan (bool)) {
func staticWorker(tasks chan (args), results chan (error), done chan (bool), shrink, verbose bool) {
task, ok := <-tasks
for ok {
results <- staticFile(task.source, task.target, task.info)
if verbose {
fmt.Println(task.source)
}
results <- staticFile(task.source, task.target, task.info, shrink)
task, ok = <-tasks
}
done <- true
}
// staticProgressIndicator watches the results channel and does a countdown. If the result channel reports an error,
// that is put into the stop channel so that staticWalk stops adding to the tasks channel.
// staticProgressIndicator watches the results channel and prints a running count. If the result channel reports an
// error, that is put into the stop channel so that staticWalk stops adding to the tasks channel.
func staticProgressIndicator(results chan (error), stop chan (error), quiet bool) (int, error) {
n := 0
t := time.Now()
var err error
for result := range results {
if result != nil {
err := result
// this stops the walker from adding more tasks
stop <- err
} else {
n++
if !quiet && n%13 == 0 {
if time.Since(t) > time.Second {
fmt.Printf("\r%d", n)
t = time.Now()
}
err, ok := <-results
for ok && err == nil {
n++
if !quiet && n%13 == 0 {
if time.Since(t) > time.Second {
fmt.Printf("\r%d", n)
t = time.Now()
}
}
err, ok = <-results
}
if ok && err != nil {
// this stops the walker from adding more tasks
stop <- err
}
return n, err
}
// staticFile is used to walk the file trees and do the right thing for the destination directory: create
// subdirectories, link files, render HTML files.
func staticFile(source, target string, info fs.FileInfo) error {
func staticFile(source, target string, info fs.FileInfo, shrink bool) error {
// render pages
if strings.HasSuffix(source, ".md") {
p, err := staticPage(source[:len(source)-3], target[:len(target)-3]+".html")
// target already has ".html" extension
p, err := staticPage(source[:len(source)-3], target)
if err != nil {
return err
}
return staticFeed(source[:len(source)-3], target[:len(target)-3]+".rss", p, info.ModTime())
return staticFeed(source[:len(source)-3], target[:len(target)-5]+".rss", p, info.ModTime())
}
// remaining files are linked unless this is a template
if slices.Contains(templateFiles, filepath.Base(source)) {
if shrink {
switch filepath.Ext(source) {
case ".jpg", ".jpeg", ".webp":
return shrinkImage(source, target, info)
}
}
// delete before linking, ignore errors
os.Remove(target)
err := os.Link(source, target)
if err == nil {
return nil
}
return os.Link(source, target)
// in case of invalid cross-device link error, copy file instead
src, err := os.Open(source)
if err != nil {
return err
}
defer src.Close()
dst, err := os.Create(target)
if err != nil {
return err
}
defer dst.Close()
_, err = io.Copy(dst, src)
return err
}
var shrinkWidth = 800
var shrinkQuality = 10
// shrink Image shrinks images down and reduces the quality dramatically.
func shrinkImage(source, target string, info fs.FileInfo) error {
file, err := os.Open(source)
if err != nil {
return err
}
defer file.Close()
img, _, err := exiffix.Decode(file)
if err != nil {
return fmt.Errorf("%s cannot be decoded", source)
}
if img.Bounds().Dx() > shrinkWidth {
res := imaging.Resize(img, shrinkWidth, 0, imaging.Lanczos) // preserve aspect ratio
// imaging functions don't return errors but empty images…
if res.Rect.Empty() {
return fmt.Errorf("%s cannot be resized", source)
}
img = res
}
dst, err := os.Create(target)
if err != nil {
return err
}
defer dst.Close()
switch filepath.Ext(source) {
case ".jpg", ".jpeg":
err = jpeg.Encode(dst, img, &jpeg.Options{Quality: shrinkQuality})
case ".webp":
err = webp.Encode(dst, img, webp.Options{Quality: shrinkQuality})
}
return err
}
// staticPage takes the filename of a page (ending in ".md") and generates a static HTML page.
@@ -200,7 +303,7 @@ func staticPage(source, target string) (*Page, error) {
return nil, err
}
p.handleTitle(true)
// instead of p.renderHtml() we do it all ourselves, appending ".html" to all the local links
// instead of p.renderHTML() we do it all ourselves, appending ".html" to all the local links
parser, hashtags := wikiParser()
doc := markdown.Parse(p.Body, parser)
ast.WalkFunc(doc, staticLinks)
@@ -210,7 +313,7 @@ func staticPage(source, target string) (*Page, error) {
}
renderer := html.NewRenderer(opts)
maybeUnsafeHTML := markdown.Render(doc, renderer)
p.Html = unsafeBytes(maybeUnsafeHTML)
p.HTML = unsafeBytes(maybeUnsafeHTML)
p.Hashtags = *hashtags
return p, write(p, target, "", "static.html")
}
@@ -221,7 +324,7 @@ func staticFeed(source, target string, p *Page, ti time.Time) error {
base := filepath.Base(source)
_, ok := index.token[strings.ToLower(base)]
if base == "index" || ok {
f := feed(p, ti, 0, 10)
f := feed(p, ti, 0, 10, ModTime)
if len(f.Items) > 0 {
return write(f, target, `<?xml version="1.0" encoding="UTF-8"?>`, "feed.html")
}
@@ -232,8 +335,7 @@ func staticFeed(source, target string, p *Page, ti time.Time) error {
// staticLinks checks a node and if it is a link to a local page, it appends ".html" to the link destination.
func staticLinks(node ast.Node, entering bool) ast.WalkStatus {
if entering {
switch v := node.(type) {
case *ast.Link:
if v, ok := node.(*ast.Link); ok {
// not an absolute URL, not a full URL, not a mailto: URI
if !bytes.HasPrefix(v.Destination, []byte("/")) &&
!bytes.Contains(v.Destination, []byte("://")) &&

View File

@@ -1,15 +1,16 @@
package main
import (
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"os"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
)
func TestStaticCmd(t *testing.T) {
cleanup(t, "testdata/static")
s := staticCli(".", "testdata/static", 2, true)
s := staticCli(".", "testdata/static", 2, "", false, false, true)
assert.Equal(t, subcommands.ExitSuccess, s)
// pages
assert.FileExists(t, "testdata/static/index.html")
@@ -34,7 +35,7 @@ And the cars so loud
`)}
h.save()
h.notify()
s := staticCli("testdata/static-feed", "testdata/static-feed-out", 2, true)
s := staticCli("testdata/static-feed", "testdata/static-feed-out", 2, "", false, false, true)
assert.Equal(t, subcommands.ExitSuccess, s)
assert.FileExists(t, "testdata/static-feed-out/2024-03-07-poem.html")
assert.FileExists(t, "testdata/static-feed-out/Haiku.html")

View File

@@ -11,11 +11,11 @@ import (
"sync"
)
// templateFiles are the various HTML template files used. These files must exist in the root directory for Oddmu to be
// able to generate HTML output. This always requires a template.
// templateFiles are the various HTML template files used. These files must exist in the root directory for Oddmu
// to be able to generate HTML output. This always requires a template.
var templateFiles = []string{"edit.html", "add.html", "view.html", "preview.html",
"diff.html", "search.html", "static.html", "upload.html", "feed.html",
"list.html"}
"sitemap.html"}
// templateStore controls access to map of parsed HTML templates. Make sure to lock and unlock as appropriate. See
// renderTemplate and loadTemplates.

View File

@@ -2,10 +2,11 @@ package main
import (
"bytes"
"github.com/stretchr/testify/assert"
"mime/multipart"
"net/http"
"testing"
"github.com/stretchr/testify/assert"
)
func TestTemplates(t *testing.T) {
@@ -21,7 +22,7 @@ Memories of cold
assert.Contains(t,
assert.HTTPBody(makeHandler(viewHandler, false, http.MethodGet), "GET", "/view/testdata/templates/snow", nil), "Skip")
// save a new view handler
html := "<body><h1>{{.Title}}</h1>{{.Html}}"
html := "<body><h1>{{.Title}}</h1>{{.HTML}}"
form := new(bytes.Buffer)
writer := multipart.NewWriter(form)
field, err := writer.CreateFormField("filename")

View File

@@ -17,7 +17,7 @@
<title>{{.Title}}</title>
<link>https://alexschroeder.ch/view/{{.Path}}</link>
<guid>https://alexschroeder.ch/view/{{.Path}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -48,9 +48,9 @@ button { background-color: #eee; color: inherit; border-radius: 4px; border-widt
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
{{range .Images}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.Html}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.HTML}}
{{end}}
</article>
{{end}}

View File

@@ -29,7 +29,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -54,7 +54,7 @@ img { max-width: 100% }
</header>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -16,7 +16,7 @@
<title>{{.Title}}</title>
<link>https://campaignwiki.org/view/{{.Path}}</link>
<guid>https://campaignwiki.org/view/{{.Path}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -39,7 +39,7 @@ img { max-width: 20% }
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
</article>
{{end}}
<p>

View File

@@ -19,7 +19,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -41,7 +41,7 @@ img { max-width: 100% }
</header>
<main>
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -37,7 +37,7 @@ button { font-size: large; background-color: #eee; color: inherit; border-radius
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
</article>
{{end}}
<p>

View File

@@ -62,7 +62,7 @@ img { max-width: 100%; margin-top: 5px }
<h1>{{.Title}}</h1>
</header>
<main>
{{.Html}}
{{.HTML}}
</main>
<footer>
<form action="/append/{{.Path}}" method="POST">

View File

@@ -16,7 +16,7 @@
<title>{{.Title}}</title>
<link>https://campaignwiki.org/view/{{.Name}}</link>
<guid>https://campaignwiki.org/view/{{.Name}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -24,7 +24,7 @@ img { max-width: 100% }
</header>
<main>
<h1>Previewing {{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<hr>
<section id="edit">

View File

@@ -40,9 +40,9 @@ button { background-color: #eee; color: inherit; border-radius: 4px; border-widt
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
{{range .Images}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.Html}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.HTML}}
{{end}}
</article>
{{end}}

View File

@@ -19,7 +19,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -43,7 +43,7 @@ img { max-width: 100% }
</header>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -10,7 +10,7 @@
<title>{{.Title}}</title>
<link>https://flying-carpet.ch/view/{{.Path}}</link>
<guid>https://flying-carpet.ch/view/{{.Path}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -38,7 +38,7 @@ img { max-width: 20% }
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
</article>
{{end}}
<p>

View File

@@ -19,7 +19,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -34,7 +34,7 @@ img { max-width: 100% }
</header>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
</body>
</html>

View File

@@ -37,9 +37,9 @@ button { background-color: #eee; color: inherit; border-radius: 4px; border-widt
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
{{range .Images}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.Html}}
<p class="image"><a href="/view/{{.Path}}"><img loading="lazy" src="/view/{{.Path}}"></a><br/>{{.HTML}}
{{end}}
</article>
{{end}}

View File

@@ -17,7 +17,7 @@
<title>{{.Title}}</title>
<link>https://transjovian.org/view/{{.Path}}</link>
<guid>https://transjovian.org/view/{{.Path}}</guid>
<description>{{.Html}}</description>
<description>{{.HTML}}</description>
<pubDate>{{.Date}}</pubDate>
{{range .Hashtags}}
<category>{{.}}</category>

View File

@@ -40,7 +40,7 @@ img { max-width: 20% }
<article lang="{{.Language}}">
<p><a class="result" href="/view/{{.Path}}">{{.Title}}</a>
<span class="score">{{.Score}}</span></p>
<blockquote>{{.Html}}</blockquote>
<blockquote>{{.HTML}}</blockquote>
</article>
{{end}}
<p>

View File

@@ -20,7 +20,7 @@ img { max-width: 100% }
<body>
<main id="main">
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -37,7 +37,7 @@ img { max-width: 100% }
</header>
<main>
<h1>{{.Title}}</h1>
{{.Html}}
{{.HTML}}
</main>
<footer>
<address>

View File

@@ -4,12 +4,13 @@ import (
"context"
"flag"
"fmt"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/google/subcommands"
"io"
"os"
"strings"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/google/subcommands"
)
type tocCmd struct {
@@ -73,8 +74,7 @@ func (p *Page) toc() Toc {
doc := markdown.Parse(p.Body, parser)
ast.WalkFunc(doc, func(node ast.Node, entering bool) ast.WalkStatus {
if !entering {
switch v := node.(type) {
case *ast.Heading:
if v, ok := node.(*ast.Heading); ok {
headings = append(headings, v)
}
}

View File

@@ -2,9 +2,10 @@ package main
import (
"bytes"
"testing"
"github.com/google/subcommands"
"github.com/stretchr/testify/assert"
"testing"
)
// ## is promoted to level 1 because there is just one instance of level 1

View File

@@ -42,7 +42,8 @@ func tokenizeWithQuotes(s string) []string {
start := -1 // valid span start if >= 0
RUNE:
for end, rune := range s {
if waitFor > 0 {
switch {
case waitFor > 0:
if rune == waitFor {
if start >= 0 {
// skip "" and the like
@@ -54,12 +55,12 @@ RUNE:
} else if start < 0 {
start = end
}
} else if unicode.IsSpace(rune) {
case unicode.IsSpace(rune):
if start >= 0 {
spans = append(spans, span{start, end})
start = ^start
}
} else {
default:
if start < 0 {
// Only check for starting quote at the beginning of a token
if IsQuote(rune) {

Some files were not shown because too many files have changed in this diff Show More