Fully support explicit tagging of .Sh and .Ss.

This fixes the offset of two lines in terminal output
and this improves HTML output by putting the id= attribute
and <a> element into the respective <h1> or <h2> element rather
than writing an additional <mark> element.

To that end, introduce node flags NODE_ID (to make the node a link
target, for example by writing an HTML id= attribute or by calling
tag_put()) and NODE_HREF (to make the node a link source, used only
in HTML output, used only to write an <a class="permalink"> element).

In particular:
* In the validator, generalize the concept of the "next node"
such that it also works before .Sh and .Ss.
* If the first argument of .Tg is empty, don't forget to complain
if there are additional arguments, which will be ignored.
* In the terminal formatter, support writing of explicit tags
for all kinds of nodes, not just for .Tg.
* In deroff(), allow nodes to have an explicit string representation
even when they aren't text nodes.  Use this for explicitly tagged
section headers.  Suprisingly, this is sufficient to make HTML
output work, without explicit code changes in the HTML formatter.
* In syntax tree output, display NODE_ID and NODE_HREF.

5 files changed, 82 insertions, 3 deletions

diff --git a/regress/usr.bin/mandoc/mdoc/Sh/Makefile b/regress/usr.bin/mandoc/mdoc/Sh/Makefile
index 213664ec7b4..4b37bb2abe0 100644
--- a/regress/usr.bin/mandoc/mdoc/Sh/Makefile
+++ b/regress/usr.bin/mandoc/mdoc/Sh/Makefile
@@ -1,11 +1,11 @@
-# $OpenBSD: Makefile,v 1.12 2020/02/27 01:25:58 schwarze Exp $
+# $OpenBSD: Makefile,v 1.13 2020/02/27 21:38:27 schwarze Exp $
 
 REGRESS_TARGETS	 = badNAME before empty emptyNAME first nohead order
 REGRESS_TARGETS	+= orderNAME paragraph parbefore parborder punctNAME
-REGRESS_TARGETS	+= subbefore transp
+REGRESS_TARGETS	+= subbefore tag transp
 LINT_TARGETS	 = badNAME before empty emptyNAME first nohead order
 LINT_TARGETS	+= orderNAME parbefore parborder punctNAME subbefore
-HTML_TARGETS	 = paragraph
+HTML_TARGETS	 = paragraph tag
 
 # groff-1.22.3 defects:
 # - .Pp before .Sh NAME causes a blank line before the header line
diff --git a/regress/usr.bin/mandoc/mdoc/Sh/tag.in b/regress/usr.bin/mandoc/mdoc/Sh/tag.in
new file mode 100644
index 00000000000..0e0e5fef16e
--- /dev/null
+++ b/regress/usr.bin/mandoc/mdoc/Sh/tag.in
@@ -0,0 +1,21 @@
+.\" $OpenBSD: tag.in,v 1.1 2020/02/27 21:38:27 schwarze Exp $
+.Dd $Mdocdate: February 27 2020 $
+.Dt SH-TAG 1
+.Os
+.Sh NAME
+.Nm Sh-tag
+.Nd tagging section headers
+.Sh DESCRIPTION
+Text in the description.
+.Ss Subsection
+BEGINTEST
+.Pp
+Text in the subsection.
+.Tg examples
+.Sh EXAMPLES
+Text introducing examples.
+.Tg example
+.Ss Subsection
+Example text.
+.Pp
+ENDTEST
diff --git a/regress/usr.bin/mandoc/mdoc/Sh/tag.out_ascii b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_ascii
new file mode 100644
index 00000000000..df0ab050a1e
--- /dev/null
+++ b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_ascii
@@ -0,0 +1,22 @@
+SH-TAG(1)                   General Commands Manual                  SH-TAG(1)
+
+NNAAMMEE
+     SShh--ttaagg - tagging section headers
+
+DDEESSCCRRIIPPTTIIOONN
+     Text in the description.
+
+   SSuubbsseeccttiioonn
+     BEGINTEST
+
+     Text in the subsection.
+
+EEXXAAMMPPLLEESS
+     Text introducing examples.
+
+   SSuubbsseeccttiioonn
+     Example text.
+
+     ENDTEST
+
+OpenBSD                        February 27, 2020                       OpenBSD
diff --git a/regress/usr.bin/mandoc/mdoc/Sh/tag.out_html b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_html
new file mode 100644
index 00000000000..9722aa8c3c5
--- /dev/null
+++ b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_html
@@ -0,0 +1,9 @@
+<p class="Pp">Text in the subsection.</p>
+</section>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="examples"><a class="permalink" href="#examples">EXAMPLES</a></h1>
+<p class="Pp">Text introducing examples.</p>
+<section class="Ss">
+<h2 class="Ss" id="example"><a class="permalink" href="#example">Subsection</a></h2>
+<p class="Pp">Example text.</p>
diff --git a/regress/usr.bin/mandoc/mdoc/Sh/tag.out_markdown b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_markdown
new file mode 100644
index 00000000000..2813f8f0d28
--- /dev/null
+++ b/regress/usr.bin/mandoc/mdoc/Sh/tag.out_markdown
@@ -0,0 +1,27 @@
+SH-TAG(1) - General Commands Manual
+
+# NAME
+
+**Sh-tag** - tagging section headers
+
+# DESCRIPTION
+
+Text in the description.
+
+## Subsection
+
+BEGINTEST
+
+Text in the subsection.
+
+# EXAMPLES
+
+Text introducing examples.
+
+## Subsection
+
+Example text.
+
+ENDTEST
+
+OpenBSD - February 27, 2020