, that can generate complex dependencies between IDs, which would either be harder to implement, or lead to infinite recursion. To prevent such problems: * for the case of <headers>: * if the <internal link> text is explicitly given, then it is used to calculate the ID of the title element and on render output * otherwise, the href argument is used for the ID and the render output. This generally works best when you use <magic links>. * for non-<headers> such as <images>, you you can link to headers, and the rendered target gets used for both ID and rendered content Here are some example: * `` = Hello \x[h2-id][the content] == H2 title {id=h2-id} `` * h1 id: `hello-the-content` * h1 render content: `Hello the content` * `` = Hello \x[h2-id] == H2 title {id=h2-id} `` * h1 id: `hello-h2-id` * h1 render content: `Hello h2-id`. Note that this is not very nice due to the `-`, but it can be made nicer without repetition with a <magic link> as in the following example * `` = Hello <blue dog> == Blue dog `` * h1 id: `hello-blue-dog` * h1 render content: `Hello blue dog` * non-header (e.g. an <image>) that links to the title of another non-header For non-headers, things are a bit more relaxed, and we can link to headers, e.g.: `` = h1 \Image[myimg.jpg] {title=my \x[h1]} `` This is allowed because OurBigBook calculates IDs in two stages: first for all headers, and only later non non-headers. What you cannot do is link to another image e.g.: `` \Image[myimg.jpg] {id=myimage1} {title=My image 1} \Image[myimg.jpg] {title=my \x[h1]} `` and there the workaround are much the same as for headers: either explicitly set the <internal link> content: `` \Image[myimg.jpg] {id=myimage1} {title=My image 1} \Image[myimg.jpg] {title=my \x[h1][My image 1]} `` or explicitly set an ID: `` \Image[myimg.jpg] {id=myimage1} {title=My image 1} \Image[myimg.jpg] {id=myimage2} {title=my \x[h1]} `` TODO both workaround are currently broken <todo/Image title with x to image with content incorrectly disallowed>, we forgot to add a test earlier on, and things inevitably broke... Should not be hard to fix though, we are just overchecking. While it is technically possible relax the above limitations and give an error only in case of loops, it would require a bit of extra work which we don't want to put in right now: https://github.com/ourbigbook/ourbigbook/issues/95[]. Furthermore, the above rules do not exclude infinite rendering loops, but OurBigBook detects such loops and gives a nice error message, this has been fixed at: https://github.com/ourbigbook/ourbigbook/issues/34 For example this would contain an infinite loop: `` \Image[myimg.jpg] {id=myimage1} {title=\x[myimage2]} \Image[myimg.jpg] {id=myimage2} {title=\x[myimage1]} `` This infinite recursion is fundamentally not technically solved: the user has to manually break the loop by providing an `x` content explicitly, e.g. in either: `` \Image[myimg.jpg] {id=myimage1} {title=\x[myimage2][my content 2]} \Image[myimg.jpg] {id=myimage2} {title=\x[myimage1]} `` or: `` \Image[myimg.jpg] {id=myimage1} {title=\x[myimage2]} \Image[myimg.jpg] {id=myimage2} {title=\x[myimage1][my content 1]} `` A closely related limitation is the simplistic approach to <\x id output format>. = Cross file internal link {parent=Internal link} = Cross file reference {synonym} = Internal cross file internal link {synonym} \Image[feature/x/hilbert-space-arrow.png] {title=<Cross file internal link>} {provider=github} {height=571} {border} Reference to the first header of another file: \OurBigBookExample[[\x[not-index\]]] Reference to a non-first header of another file: \OurBigBookExample[[\x[h2-in-not-the-index\]]] To make toplevel links cleaner, if the target header is the very first element of the other page, then the link does not get a fragment, e.g.: `\x[not-index]` rendered as: `` <a href="not-index" `` and not: `` <a href="not-index#not-index" `` while `\x[h2-in-not-the-index]` is rendered with the fragment: `` <a href="not-index#h2-in-not-the-index" `` Reference to the first header of another file that is a second inclusion: \OurBigBookExample[[\x[included-by-not-index\]]] Reference to another header of another file, with <\x full argument>[`full`]: \OurBigBookExample[[\x[h2-in-not-the-index]{full}.]] Note that when `full` is used with references in another file in <--embed-includes>[multi page mode], the number is not rendered as explained at: <\x full argument in cross file internal links>{full}. Reference to an image in another file: \OurBigBookExample[[\x[image-not-index-xi]{full}.]] Reference to an image in another file: \OurBigBookExample[[\x[image-figure-in-not-the-index-without-explicit-id]{full}.]] Remember that the <the toplevel header>[ID of the toplevel header] is automatically derived from its file name, that's why we have to use: \OurBigBookExample[[\x[not-index\]]] instead of: `` \x[not-the-index] `` Reference to a subdirectory: \OurBigBookExample[[ \x[subdir] \x[subdir/h2] \x[subdir/notindex] \x[subdir/notindex-h2] ]] Implemented at: https://github.com/ourbigbook/ourbigbook/issues/116 Reference to an internal header of another file: <h2 in not the index>. By default, That header ID gets prefixed by the ID of the top header. When using <--embed-includes> mode, the cross file internal links end up pointing to an ID inside the current HTML element, e.g.: `` <a href="#not-index"> `` rather than: `` <a href="not-index.html/#not-index"> `` This is why IDs must be unique for elements across all pages. = Cross file internal link internals {parent=Cross file internal link} {title2=`db.sqlite3`} = Cross file reference internals {synonym} = ID database {c} {synonym} When running in Node.js, OurBigBook dumps the IDs of all processed files to a `_out/db.sqlite3` file in <the out directory>, and then reads from that file when IDs are needed. When converting under a directory that contains <ourbigbook.json>, `_out/db.sqlite3` is placed inside the same directory as the `ourbigbook.json` file. If there is no `ourbigbook.json` in parent directories, then `_out/db.sqlite3` is placed in the current working directory. These follows the principles described at: <the current working directory does not matter when there is a ourbigbook.json>. `db.sqlite3` is not created or used when handling input from <stdin>. When running in the browser, the same JavaScript API will send queries to the server instead of a local SQLite database. To inspect the ID database to debug it, you can use: `` sqlite3 _out/db.sqlite3 .dump `` It is often useful to dump a single table, e.g. to dump the `ids` table: `` sqlite3 _out/db.sqlite3 '.dump ids' `` and one particularly important query is to dump a list of all known IDs: `` sqlite3 _out/db.sqlite3 'select id from ids' `` You can force `ourbigbook` to not use the ID database with the <--no-db> command line option = Link to IDs, not URL path {parent=Cross file internal link} This section describes the philosophy of <internal links>. In many static website generators, you just link to URL specific paths of internal headers. In OurBigBook, <internal links> point to IDs, not paths. For example, suppose "Superconductivity" is a descendant of "Condensed Matter Physics", and that the source for both is located at `condensed-matter-physics.bigb`, so that both appear on the same .html page `condensed-matter-physics.html`. When linking to Superconductivity from an external page such as `statistical-physics.bigb` you write just `<superconductivity>` and NOT `<condensed-matter-physics#superconductivity>`. OurBigBook then automatically trakcs where superconductivity is located and produces `href="condensed-matter-physics#superconductivity"` for you. This is important because on a static website, the location of headers might change. E.g. if you start writing a lot about superconductivity you would eventually want to split it to its own page, `superconductivity.html` otherwise page loads for `condensed-matter-physics.html` would become too slow as that file would become too large. But if your links read `<condensed-matter-physics#superconductivity>`, and all links would break when you move things around. So instead, you simply link to the ID `<superconductivity>`, and ourbigbook renders links correctly for you wherever the output lands. When moving headers to separate pages, it is true that existing links to subheaders will break, but that simply cannot be helped. Large pages must be split into smaller ones. The issue can be mitigated in the following ways: * <split headers>, which readers will eventually understand are better permalinks * <JavaScript redirect to split on missing ID>, which automatically redirect `condensed-matter-physics#superconductivity` to `superconductivity`, potentially hitting a <split header> if the current page does not contain the HTML ID `superconductivity`. For <OurBigBook Web>, this is even more important, as we have <dynamic article trees>, so every header can appear on top. If you really want to to use scopes, e.g. enforce the ID of "superconductivity" to be "condensed-matter-physics/superconductivity", then you can use the <scope> feature. However, this particular case would likely be a bad use case for that feature. You want your IDs to be as short as possible, which causes less need for refactoring, and makes <topics> on <OurBigBook Web> more likely to have matches from other users. \Image[feature/x/hilbert-space-arrow.png] {title=<Cross file internal link>} {disambiguate=Link to IDs, not URL path} {provider=github} {height=571} {border} = Internal link title link removal {parent=Cross file internal link} = Cross reference title link removal {synonym} If the target `title` argument contains a link from either another <internal links> or a regular <link>[external hyperlink], OurBigBook automatically prevents that link from rendering as a link when no explicit body is given. This is done because https://stackoverflow.com/questions/9882916/are-you-allowed-to-nest-a-link-inside-of-a-link[nested links are illegal in HTML], and the result would be confusing. This use case is most common when dealing with media such as <images>. For example in: `` = afds \x[image-aa-zxcv-lolol-bb] == qwer \Image[Tank_man_standing_in_front_of_some_tanks.jpg] {title=aa \x[zxcv][zxcv] \a[http://example.com][lolol] bb} == zxcv `` the `\x[image-aa-zxcv-lolol-bb]` renders something like: `` <a href="#image-aa-zxcv-lolol-bb">aa zxcv lolol bb</a> `` and not: `` <a href="#image-aa-zxcv-lolol-bb">aa <a href="zxcv">zxcv</a> <a href="http://example.com">lolol</a> bb</a> `` Live example: \OurBigBookExample[[ This is a nice image: \x[image-aa-zxcv-lolol-bb]. \Image[Tank_man_standing_in_front_of_some_tanks.jpg] {title=aa \x[internal-link-title-link-removal][zxcv] \a[http://example.com][lolol] bb} ]] = `\x` arguments {parent=Internal link} = `\x` `c` argument {parent=x arguments} Capitalizes the first letter of the target title. For more details, see: <internal link title inflection>{full}. = `\x` `child` argument {parent=x arguments} {tag=Disabled macro argument} Setting the `child` <boolean argument> on a <internal link> to a header as in: `` \x[my-header]{child} `` makes that header show up on the list of extra parents of the child. This argument is deprecated in favor of the <H tag argument>. This allows a section to have multiple parents, e.g. to include it into multiple categories. For example: `` = Animal == Mammal === Bat === Cat == Flying animal These animals fly: * \x[bat]{child} These animals don't fly: * \x[cat] `` would render something like: `` = Animal == Mammal === Bat (Parent section: Mammal) (Tags: Flying animal) === Cat (Parent section: Mammal) == Flying animal (Parent section: Animal) These animals fly: * \x[bat] These animals don't fly: * \x[cat] `` so note how "Bat" has a list of tags including "Flying animal", but Cat does not, due to the `child`. This property does not affect how the <table of contents> is rendered. We could insert elements sections there multiple times, but it has the downside that browser Ctrl + F searches would hit the same thing multiple times on the table of contents, which might make finding things harder. `` == My title{id=my-id} Read this \x[my-id][amazing section]. `` If the second argument, the <`content` argument>, is not present, it expand to the header title, e.g.: `` == My title{id=my-id} Read this \x[my-id]. `` is the same as: `` == My title{id=my-id} Read this \x[my-id][My title]. `` = Secondary children {parent=x child argument} The term refers to sections that have a parent/child relationship via either of the: * <\x child argument> * <\x parent argument> * <\H child argument> * <\H tag argument> rather than via the usual <header> hierarchy. Secondary children show up for example on the <tagged metadata section>, but not on the <table of contents>, which is what the header hierarchy already shows. Secondary children are normally basically used as "tags": a header such as `Bat` can be a direct child of `Mammal`, and a secondary child of `Flying animal`, or vice versa. Both `Mammal` and `Flying animal` are then basically ancestors. But we have to chose one main ancestor as "the parent", and other secondary ancestors will be seen as tags. This option first does <ID target from title> conversion on the argument, so you can e.g. keep any spaces or use capitalization in the title as in: `` = Animal == Flying animal {child=Big bat} == Big bat `` TODO the fact that this transformation is done currently makes it impossible to use "non-standard IDs" that contain spaces or uppercase letters. If someone ever wants that, we could maybe add a separate argument that does not do the expansion e.g.: `` = Animal == Flying animal {childId=Big bat} == Big bat {id=Big bat} `` but definitely the most important use case is having easier to type and read source with the standard IDs. = `\x` `file` argument {parent=x arguments} Allows to link to <headers> with the <`\H` `file` argument>, e.g.: `` = My header Check out this amazing file: <path/to/myfile.txt>{file} == path/to/myfile.txt `` Some live demos follow: \OurBigBookExample[[ \x[file_demo]{file} ]] \OurBigBookExample[[ \x[file_demo/file_demo_subdir]{file} ]] \OurBigBookExample[[ \x[file_demo/file_demo_subdir/hello_world.js]{file} ]] \OurBigBookExample[[ \x[file_demo/my.bin]{file} ]] \OurBigBookExample[[ \x[Tank_man_standing_in_front_of_some_tanks.jpg]{file} ]] \OurBigBookExample[[ \x[https://www.youtube.com/watch?v=YeFzeNAHEhU]{file} ]] = `\x` `full` argument {parent=x arguments} To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` <boolean argument> to the <internal link>, for example: \OurBigBookExample[[\x[x-full-argument]{full}.]] `{full}` is not needed for <internal links> to most macros besides <headers>, which use `full` by default as seen by the `default_x_style_full` macro property in <--help-macros>. This is for example the case for <images>. You can force this to be disabled with `{full=0}`: \OurBigBookExample[[Compare \x[image-my-test-image]{full=0} vs \x[image-my-test-image]{full=1}.]] = `\x` `full` argument in cross file internal links {parent=x full argument} For example in the following <cross file internal link>: \OurBigBookExample[[\x[h2-in-not-the-index]{full}.]] we get just something like: `` Section "h2 in not the index" `` instead of: `` Section 1.2 "h2 in not the index" `` This is because the number "Section 1.2" might already have been used in the current page, leading to confusion. = `\x` `magic` argument {parent=x arguments} {tag=Boolean argument} = Magic link {synonym} This argument makes writing many <internal links> more convenient, and it was notably introduced because it serves as the sane version of <shorthand internal links>. If given e.g. as in: `` = Internal reference \x[Internal references]{magic} `` the link treated magically as follows: * content capitalization and pluralization are detected from the string, and implicitly set the <\x c argument> and <\x p argument>. In the example: * `{c}` capitalization is set because `Internal references` starts with an upper case character `I` * `{p}` pluralization is set because `Internal references` ends in a plural word In this simple example, the content therefore will be exactly `Internal references` as in the source. But note that this does not necessarily need to be the case, e.g. if we had done: `` \x[Internal Reference]{magic} `` then the content would be: `` Internal reference `` without capital `R`, i.e. everything except capitalization and pluralization is ignored. This forgiving way of doing things means that writers don't need to remember the exact ideal capitalization of everything, which is very hard to remember. It also means that any more complex elements will be automatically rendered as usual, e.g. if we had: `` = \i[Internal] reference \x[internal reference]{magic} `` then the output would still contain the `<i>` italic tag. If we had a scope as in `\x[my scope/Internal references]`, then each scope part is checked separately. E.g. in this case we would have upper case `Internal references`, even though `my scope` is lowercase, and so `{c}` would be set. * the ID is calculated as follows: * <automatic ID from title> conversion is performed, with you exception: forwards slashs `/` are kept, in order to make <scopes> work. In our case, there aren't any slashes `/`, so it just gives `internal-references`. But if instead we had e.g.: `\x[my scope/internal reference]{magic}`, then we would reach `my-scope/internal-reference` and not `my-scope-internal-reference`. * if there is a match to an existing ID use it. `internal-references` in the plural does not match, so go to the next step * if the above failed, try singularizing the last word as in the <\x p argument> with `p=0` before doing <automatic ID from title> conversion. This gives `internal-reference`, which does exist, and so we use that. There may be some cases where you might still want to use <internal link title inflection> however, see: <inflection vs magic>{full}. = Shorthand magic link {parent=`\x` `magic` argument} {tag=Shorthand macro syntax} = Insane magic link {synonym} A <magic link> can be created more succinctly by surrounding the link with "angle brackets" (`<>`), e.g.: `` <Partial derivative> `` is equivalent to: `` \x[Partial derivative]{magic} `` = `\x` `parent` argument {parent=x arguments} {tag=Disabled macro argument} The `parent` argument is exactly like the <\x child argument>, but it reverses the direction of the parent/child relation. This argument is deprecated in favor of the <H tag argument>. = `\x` `ref` argument {parent=x arguments} The `ref` argument of `\x` marks the link as reference, e.g.: `` Trump said this and that.\x[donald-trump-access-hollywood-tape]{ref} = Donald Trump Access Hollywood tape `` renders something like: `` Trump said this and that.<a href="donald-trump-access-hollywood-tape">*</a> `` This could currently be replicated without `ref` by just using: `` Trump said this and that.\x[donald-trump-access-hollywood-tape][*] `` but later on we might add more precise reference fields like the page of a book or date fetched as Wikipedia supports. Implemented at: https://github.com/ourbigbook/ourbigbook/issues/137 = `\x` `topic` argument {parent=x arguments} {title2=`<#Black holes>`} {tag=Boolean argument} = Topic link {synonym} {title2} If true, then the target of a this link is called a "topic link" and gets treated specially, pointing to an external <OurBigBook Web topic> rather than a header defined in the current project. For example, when rendering a <static website>, a link such as: `` \x[Albert Einstein]{topic} `` would produce output similar to: `` \a[https://ourbigbook.com/-/topic/john-smith][John Smith] `` e.g.: \OurBigBookExample[[ \x[Albert Einstein]{topic} ]] This allows static website creators to easily link to topics they might not have already written about which others may have covered. The <OurBigBook Web> instance linked to can be configured with <ourbigbook.json/web/host>. Those links also work on <OurBigBook Web> rendering of course, and point to the current Web instance. = Shorthand topic link {parent=`\x` `topic` argument} {tag=Shorthand macro syntax} {title2=`<#Albert Einstein>`} = Insane topic link {synonym} If an <shorthand magic link> starts with a hash sign (`#`), then it is converted to a <topic link> instead of a <magic link>. For example: \OurBigBookExample[[ <#Albert Einstein> ]] is equivalent to: `` \x[Albert Einstein]{topic} `` = Shorthand topic links with a single word {parent=Shorthand topic link} {tag=Shorthand macro syntax} {title2=`#star`} = Insane topic links with a single word {synonym} If an <shorthand topic link> is made up of a single word then it can be written in the following even succincter notation, without the need for angle brackets: \OurBigBookExample[[ I like #dogs ]] is equivalent to: `` I like <#dogs> `` Word separation is defined analogously to <shorthand link parsing rules>, i.e.: * `#` can start from anywhere, including the middle of words, e.g.: \OurBigBookExample[[ abc#mytopic ]] would produce a link immediately preceded by the characters `abc`. * `#` ends at any <shorthand link termination character>, e.g.: * Topic is `mytopic`: \OurBigBookExample[[ #mytopic is cool ]] * Topic is `mytopic,` with the comma: \OurBigBookExample[[ #mytopic, is cool ]] So you likely would have had instead used the sane syntax in this case with `<#mytopic>, is cool` to avoid that. = Topic link pluralization {parent=`\x` `topic` argument} Unlike local links, it is not possible to automatically determine the exact pluralization of a <topic link> because: * it would require communicating with the <OurBigBook Web> API, which we could in principle do, but we would rather not have static builds depend on Web instances * topics can be written by multiple authors, and there could be both plural and singular versions of each topic ID, which makes it hard to determine which one is "correct" Therefore, it is up to authors to specifically specify the desired pluralization of their topic links: * by default, topic IDs are automatically singularized, e.g.: `` <#Many Dogs> `` renders something like: `` \a[https://ourbigbook.com/-/topic/many-dog][Many Dogs] `` * to prevent this automatic singularization, use <`\x` `p` argument> with `{p}`, e.g.: `` <#Many Dogs>{p} `` renders something like: `` \a[https://ourbigbook.com/-/topic/many-dogs][Many Dogs] `` This is unfortunately always necessary for uncountable nouns such as "mathematics": \OurBigBookExample[[ I like #mathematics{p} ]] since our underlying pluralization library <blakeembrey pluralize> cannot handle uncountable nouns reliably. = `\x` `p` argument {parent=x arguments} {tag=Boolean argument} Pluralizes or singularizes the last word of the target title. For more details, see: <internal link title inflection>{full}. = OurBigBook Markup syntax {c} {parent=OurBigBook Markup} = Macro shorthand syntax {parent=OurBigBook Markup syntax} = Shorthand macro syntax {synonym} = Shorthand syntax {synonym} = Shorthand {synonym} = Insane macro shortcut {synonym} = Insane syntax {synonym} = Insane {synonym} = Insanity {synonym} = Implicit {synonym} = Implicit syntax {synonym} = Explicit {synonym} = Explicit macro {synonym} Certain commonly used macros have a shorthand macro syntax that do not start with backslash (`\`). It is a form of <#syntactic sugar>. For example, you could write <code blocks> as: \OurBigBookExample[[[ I like my code: \C[[ print("hello world") ]] ]]] but that is very verbose and annoying to read and write. Therefore, in addition to the <explicit> `\C` syntax, most people will prefer the backtick shorthand syntax: \OurBigBookExample[[[ I like my code: `` print("hello world") `` ]]] Originally, <design goals>[Ciro wanted to avoid those], but they just feel too good to avoid. Every shorthand syntax does however have an equivalent sane syntax. Our style recommendation is: use the shorthand version which is shorter, unless you have a specific reason to use the sane version. Shorthand in our context does not mean worse. It just mean "harder for the computer to understand". But it is more important that humans can understand in the first place! It is find to make the computer work a bit more for us when we are able to. = Macro with shorthand syntax {parent=Macro shorthand syntax} = Macros with shorthand syntax {synonym} = Macro with insane shortcut {synonym} = Macros with insane shortcut {synonym} * <paragraph>{full}: `\n\n` (double newline) * <link>{full}: `a http://example.com b` (space followed by `http://`) * <internal link>{full}: `<Internal links>` (angle brackets), see: <shorthand internal link>{full} * <mathematics>{full}: `$`, described at: <code and math shorthands> * <code block>{full}: \c[[`]], described at: <code and math shorthands> * <list>{full}: `* ` and indentation * <table>{full}: `|| `, `| ` and indentation * <quotation>{full}: `> ` = Code and math shorthands {parent=Macro with shorthand syntax} = Insane code and math shortcuts {synonym} The shorthand code and math shortcuts work very analogously and are therefore described together in this section. The shorthand inline code syntax: \OurBigBookExample[[a `b c` d]] and is equivalent to the sane: `` a \c[[b c]] d `` The shorthand block code: \OurBigBookExample[[ a `` b c `` d ]] and is equivalent to the sane: `` a \C[[ b c ]] d `` = Shorthand syntax extra arguments {parent=Macro with shorthand syntax} = Insane macro shortcut extra arguments {synonym} Shorthand arguments always work by abbreviating: * the macro name * one or more of its positional arguments, which are fixed as either <literal arguments>[literal or non-literal] for a given shorthand construct This means that you can add further arguments as usual. For example, an shorthand code block with an id can be written as: `` a `b c`{id=ef} g `` because that is the same as: \OurBigBookExample[[a \c[b c]{id=ef} g]] So we see that the `b c` argument is the very first argument of `\c`. Extra arguments must come after the shorthand opening, e.g. the following does not work: `` a {id=ef}`b c` g `` This restriction things easy to parse for humans and machines alike. = Escapes in macro shorthands {parent=Macro with shorthand syntax} = Escapes in insane macro shortcuts {synonym} Literal backticks and dollar signs can be produced witha backslash escape as in: \OurBigBookExample[[a \` \$ b]] It is not possible to escape backticks (\c[[`]]) inside an shorthand inline code, or dollar signs (`$`) in shorthand math. The design reason for that is because multiple backticks produce block code. The upside is that then you don't have to escape anything else, e.g. backslashes (`\`) are rendered literally. The only way to do it is to use the sane syntax instead: \OurBigBookExample[[[ a \c[[b ` c]] d a \m[[\sqrt{\$4}]] d ]]] Within block code and math, you can just add more separators: \OurBigBookExample[[ ``` code with two backticks `` nice ``` ]] = Macro identifier {parent=OurBigBook Markup syntax} <OurBigBook Markup> macro identifiers can consist of the following letters: * `a-z` lowercase * `A-Z` uppercase * `0-9` Since underscores `_` or hyphens `=` are not allowed, https://en.wikipedia.org/wiki/Camel_case[camel case] macro names are recommended, e.g. for <\OurBigBookExample> we use the name: `` OurBigBookExample `` = Macro argument {parent=OurBigBook Markup syntax} = Macro argument syntax {synonym} = Positional vs named arguments {parent=Macro argument} {title2=`[...]` vs `{key=...}`} Every argument in OurBigBook is either positional or named. For example, in a <header> definition with an ID: `` = My asdf {id=asdf qwer} {scope} `` which is equivalent to the <explicit> version: `` \H[1][My asdf] {id=asdf qwer} {scope} `` we have: * two positional argument: `[1]` and `[My asdf]`. Those are surrounded by square brackets `[]` and have no name * two named arguments: `{id=asdf qwer}` and `{scope}`. The first one has name `id`, followed by the separator `=`, followed by the value `asdf qwer`. The separator `=` always is optional. If not given, it is equivalent to an empty value, e.g.: `` {id=} `` is the same as: `` {id} `` You can determine if a macro is positional or named by using <--help-macros>. Its output contains something like: `` "h": { "name": "h", "positional_args": [ { "name": "level" }, { "name": "content" } ], "named_args": { "id": { "name": "id" } "scope": { "name": "scope" } }, `` and so we see that `level` and the <`content` argument> are positional arguments, and `id` and `scope` are named arguments. Generally, positional arguments are few (otherwise it would be hard to know which is which is which), and are almost always used for a given element so that they save us from typing the name too many times. The order of positional arguments must of course be fixed, but named arguments can go anywhere. We can even mix positional and named arguments however we want, although this is not advised for clarity. The following are therefore all equivalent: `` \H[1][My asdf]{id=asdf qwer}{scope} \H[1][My asdf]{scope}{id=asdf qwer} \H{id=asdf qwer}{scope}[1][My asdf] \H{scope}[1]{id=asdf qwer}[My asdf] `` Just like named arguments, positional arguments are never mandatory. = Positional argument {parent=Positional vs named arguments} See: <positional vs named arguments>{full}. = Positional argument default values {parent=Positional argument} Most positional arguments will default to an empty string if not given. However, some positional arguments can have special effects if not given. For example, an anchor with the first positional argument present (the URL), but not the second positional argument (the link text) as in: \OurBigBookExample[[\a[http://example.com\]]] has the special effect of generating automatic links as in: `` \a[http://example.com][http://example.com] `` This can be contrasted with named arguments, for which there is always a default value, notably for <boolean arguments>. See also: <link>{full}. = Mandatory positional arguments {parent=Positional argument} Some positional arguments are required, and if not given OurBigBook reports an error and does not render the node. This is for example the `level` of a <header>. These arguments marked with the `mandatory: true` <--help-macros> argument property. = Named argument {parent=Positional vs named arguments} See: <positional vs named arguments>{full}. = Macro argument type {parent=Macro argument} = Boolean argument {parent=Macro argument type} Name arguments marked in <--help-macros> as `boolean: true` must either: * take no value and no `=` sign, in which case the value is implicitly set to `1` * take value exactly `0` or `1` * not be given, in which case a custom per-macro default is used. That value is the `default` from <--help-macros>, or `0` if such default is not given For example, the <\x full argument> of <internal links> is correctly written as: \OurBigBookExample[[\x[boolean-argument]{full}]] without the `=` sign, or equivalently: \OurBigBookExample[[\x[boolean-argument]{full=1}]] The `full=0` version is useful in the case of reference targets that unlike <headers> expand the title on the <internal link> by default, e.g. <images>: \OurBigBookExample[[\x[boolean-argument]{full=1}]] The name "boolean argument" is given by analogy to the https://stackoverflow.com/questions/16109358/what-is-the-correct-readonly-attribute-syntax-for-input-text-elements/24588427#24588427["boolean attribute" concept in HTML5]. = Date argument {parent=Macro argument type} The value of this argument must be a valid date string as accepted by JavaScript `new Date()` e.g. 2000-01-02. = Positive nonzero integer argument {parent=Macro argument type} Positive nonzero integer arguments accept only the characters `[0-9]` as their input, and 0 may not be the first character. If anything else is present, an error is raised. = Common argument {c} {parent=Macro argument} <Common arguments> are argument names that are present in all macros. = `id` argument {parent=Common argument} Explicitly sets the ID of a macro. In <OurBigBook Markup>, every single macro has an ID, which can be either: * explicit: extracted from some input given by the user, either the <`id` argument> or the <`title` argument>. Explicit IDs can be referenced in <Internal links> and must be unique * implicit: automatically generated numerical ID. Implicit IDs cannot be referenced in <internal links> and don't need to be unique. Their primary application is generating on hover links next to everything you hover, e.g. arbitrary paragraphs. The most common way to assign an ID is implicitly with <automatic ID from title> conversion for macros that have a <`title` argument>. The <`id` argument> allows to either override the <automatic ID from title>, or provide an explicit ID for elements that don't have a <`title` argument>. = `disambiguate` argument {parent=Common argument} = `\H` `disambiguate` argument {synonym} Sometimes the short version of a name is ambiguous, and you need to add some extra text to make both its title and ID unique. For example, the word "Python" could either refer to: * the programming language: https://en.wikipedia.org/wiki/Python_(programming_language) * the genus of snakes: https://en.wikipedia.org/wiki/Python_(genus) The `disambiguate` <named argument> helps you deal more neatly with such problems. Have a look at this example: `` My favorite snakes are \x[python-genus]{p}! My favorite programming language is \x[python-programming-language]! \x[python-genus]{full} \x[python-programming-language]{full} = Python {disambiguate=genus} {parent=disambiguate-argument} = Python {c} {disambiguate=programming language} {parent=disambiguate-argument} {title2=.py} {wiki} `` from which we observe how `disambiguate`: * gets added to the ID after conversion following the same rules as <automatic ID from title> * shows up on the header between parenthesis, much like Wikipedia, as well as in <\x full argument>[`full` internal links] * does not show up on non-`full` references. This makes it much more likely that you will be able to reuse the title automatically on an <internal link> without the <`content` argument>: we wouldn't want to say "My favorite programming language is Python (programming language)" all the time, would we? * gets added to the default <\H wiki argument> inside parenthesis, following Wikipedia convention, therefore increasing the likelihood that you will be able to go with the default Wikipedia value Besides disambiguating headers, the `disambiguate` argument has a second related application: disambiguating IDs of images. For example: \OurBigBookExample[[ \x[image-the-title-of-my-disambiguate-image]{full=0} \x[image-the-title-of-my-disambiguate-image-2]{full=0} \x[image-the-title-of-my-disambiguate-image]{full} \x[image-the-title-of-my-disambiguate-image-2]{full} \Image[Tank_man_standing_in_front_of_some_tanks.jpg] {title=The title of my disambiguate image} \Image[Tank_man_standing_in_front_of_some_tanks.jpg] {title=The title of my disambiguate image} {disambiguate=2} ]] Note that unlike for headers, `disambiguate` does not appear on the title of images at all. It serves only to create an unique ID that can be later referred to. Headers are actually the only case where `disambiguate` shows up on the visible rendered output. We intend on making this application obsolete however with: This use case is even more useful when `title-from-src` is enable by default for the <ourbigbook.json/media-providers> entry, so you don't have to repeat titles several times over and over. = JavaScript interface for arguments {c} {parent=Macro argument} The JavaScript interface sees arguments as follows: `` function macro_name(args) `` where args is a dict such that: * optional arguments have the key/value pairs explicitly given on the call * mandatory arguments have a key documented by the API, and the value on the call. For example, the link API names its arguments `href` and `text`. = Literal arguments {parent=Macro argument} {title2=`[[...]]` and `{{key=...}}`} = Literal argument {synonym} Arguments that are opened with more than one square brackets `[` or curly braces `{` are literal arguments. In literal arguments, OurBigBook is not parsed, and the entire argument is considered as text until a corresponding close with the same number of characters. Therefore, you cannot have nested content, but it makes it extremely convenient to write <code blocks> or <mathematics>. For example, a multiline code block with double open and double close square brackets inside can be enclosed in triple square brackets: \OurBigBookExample[[[ A literal argument looks like this in OurBigBook: \C[[ \C[ A multiline code block. ] ]] And another paragraph. ]]] The same works for inline code: \OurBigBookExample[[[The program \c[[puts("]");]] is very complex.]]] Within literal blocks, only one thing can be escaped with backslashes are: * leading open square bracket `[` * trailing close square bracket `]` The rule is that: * if the first character of a literal argument is a sequence of backslashes (`\`), and it is followed by another argument open character (e.g. `[`, remove the first `\` and treat the other characters as regular text * if the last character of a literal argument is a `\`, ignore it and treat the following closing character (e.g. `]`) as regular text See the following open input/output pairs: `` \c[[\ b]] <code>\ b</code> \c[[\a b]] <code>\a b</code> \c[[\[ b]] <code>[ b</code> \c[[\\[ b]] <code>\[ b</code> \c[[\\\[ b]] <code>\\[ b</code> `` and close examples: `` \c[[a \]] <code>a \</code> \c[[a \]]] <code>a ]</code> \c[[a \\]]] <code>a \]</code> `` = Newline removal {parent=Macro argument} OurBigBook ignores most single newlines present in your source code. This allows authors to write much more readable <OurBigBook Markup> source. This removal includes things like: * <argument newlines between arguments removal> * when the newline is surrounded by one or two <block macros>, for example: `` This quote:\Q[My quote]is very cool `` is the same as: `` This quote: \Q[My quote] is very cool `` because <quotation> is a <block macro>. * <argument leading and trailing newline removal> The exception is when the newline is placed between two <inline macros>, where it generates an explicit <line break>. This is particularly useful for poetry, for example: \OurBigBookExample[[ Roses are red Violets are blue ]] and produces HTML like: `` Roses are red<br>Violets are blue `` Double newlines generate <paragraphs>, and triple newlines are forbidden except in <literal arguments>. = Argument leading and trailing newline removal {parent=Newline removal} = Argument leading newline removal {synonym} If the very first or very last character of an argument is a newline, then that character is ignored if it would be part of a regular plaintext node. For example: `` \C[[ a b ]] `` generates something like: `` <pre><code>a b</code></pre> `` instead of: `` <pre><code> a b </code></pre> `` This is extremely convenient to improve the readability of code blocks and similar constructs. The newline is however considered if it would be part of some <macro shorthand syntax>. For example, we can start an <list>[shorthand list] inside a <quotations> as in: \OurBigBookExample[[ \Q[ * a * b ] ]] where the shorthand list requires a leading newline `\n* ` to work. That newline is not ignored, even though it comes immediately after the `\Q[` opening. = Argument newlines between arguments removal {parent=Newline removal} The macro name and the first argument, and any two consecutive arguments, can be optionally separated by exactly one newline character, e.g.: `` \H [2] {scope} [Design goals] `` is equivalent to: `` \H[2]{scope}[Design goals] `` which is also equivalent to: `` \H[2]{scope} [Design goals] `` This allows to greatly improve the readability of long argument lists by having them one per line. There is one exception to this however: inside an <header>[shorthand header], any newline is interpreted as the end of the shorthand header. This is why the following works as expected: `` == My header 2 `some code` {id=asdf} `` and the `id` gets assigned to the header rather than the trailing code element. = Document trailing newline removal {parent=Newline removal} If the document ends one newline, it is ignored. If it is two or more, then that generates an error. = CRLF {parent=Macro argument} OurBigBook treats CRLF newlines exactly as LF, to help with Windows interoperability. = Escape characters {parent=Macro argument} Every character that cannot be a <macro identifier> can be escaped with a backslash `\`. If you try to escape a macro identifier it of course treats the thing as a macro instead and fails, e.g. in `\a` it would try to use a macro called `\a`, not escape the character `a`. For some characters, escaping or not does not make any difference because they don't have any meaning to <OurBigBook Markup>, e.g. currently `%` is always the exact same as `\%`. But in <literal arguments>[non-literal macro arguments], you have to use a backslash to escape the following if you want them to not have any magical meaning: * `\`: backslashes start macros * `\[` and `\]`: open and close <positional vs named arguments>[positional macro arguments] * `\{` and `\}`: open and close <positional vs named arguments>[optional macro arguments] * escapes for <macros with shorthand syntax>: * `<` (open angle brackets, less than sign): <macro shorthand syntax> for <shorthand internal links> * `$` (dollar sign): <macro shorthand syntax> for <mathematics> * \c[[`]] (backtick): <macro shorthand syntax> for <code blocks> * `#` (hash): <shorthand topic links> Furthermore, only at: * at the start of the document * after a newline * at the start of a new argument you must also escape the following <macros with shorthand syntax>: * `* ` for shorthand <lists> * `| ` and `|| ` for shorthand <tables> * `= ` for shorthand <headers> The escape rules for literal arguments are described at: <literal arguments>{full}. This is good for short arguments of regular text, but for longer blocks like <code blocks> or <mathematics>, you may want to use <literal arguments> = Macro argument property {parent=Macro argument} Each <macro argument> can have certain properties associated to it. These properties have programmatic effects, and allow users and developers to more easily understand and create new macro arguments. = Disabled macro argument {parent=Macro argument property} Some macro arguments are disabled by default. These are typicially arguments which felt like a good idea one day, but which we ended up regretting. They can be enabled via <ourbigbook.json> options TODO, but doing so will make the project incompatible with <OurBigBook Web>, so it is not advised. = `cannotContains` macro argument {parent=Macro argument property} A set of strings. Each one is a macro that this argument cannot contain. The prototypical use case of this is to prevent nesting of HTML `<a>` elements which is illegal in HTML and doesn't make much sense anyways, e.g.: `` \a[http://example.com][\a[http://example2.com]] `` = `empty` macro argument {parent=Macro argument property} An `empty` macro argument must always be empty, or else an error is raised. The main application of such arguments is to allow macros that normally take no arguments to be immediately followed by text, e.g.: \OurBigBookExample[[ ab\br[]cd ]] = `inlineOnly` {parent=Macro argument property} A macro argument property that is `inlineOnly` can only contain <inline macros>. If any <block macros> present in the argument or its descendants, will lead to a conversion error. Some notable rules: * <title arguments> are alwayws `inlineOnly` * all arguments of <inline macros> are `inlineOnly` There are two main rationales for enforcing these rules: * the HTML `h1` - `h6` header HTML elements can only contain phrasing content (analogout to our <inline macros>) for the HTML to be valid. We could chose to use styled `div`s instead of `h` elements, but this could have a negative SEO impact. All other HTML elements could be replaced by `div`s without issue however, the problem really is only `h`. * on <OurBigBook Web>, where multiple users are working together and many titles from multiple users show on index pages, it is saner to be more restrictive on what is allowed on titles and to prevent visually very large things from being added in order to prevent bad actors or accidents from disrupting other users too much = `multiple` argument {parent=Macro argument property} By default, arguments can be given only once. However, arguments with the `multiple` <macro argument property> set to `true` can be given multiple times, and each time the argument is given, the new value is appended to a list containing all the values. An example is the <H tag argument>. <internals API>[Internally], multiple is implemented by creating a new level in the <abstract syntax tree>, and storing each argument separately under a newly generated dummy nodes as in: `` AstNode: H AstArgument: child AstNode: Comment AstArgument: content AstNode: plaintext AstNode: x AstNode: Comment AstArgument: content AstNode: plaintext AstNode: x `` = `notEmpty` macro argument {parent=Macro argument property} This property is the opposite of the <empty macro argument>: the argument cannot be empty or else that is error. = `remove_whitespace_children` {parent=Macro argument property} In HTML, certain elements such as `<ul>` cannot have any `text` nodes in them, and any whitespace is ignored, see https://stackoverflow.com/questions/2161337/can-we-use-any-other-tag-inside-ul-along-with-li/60885802#60885802[]. A similar concept applies to OurBigBook, e.g.: `` \Ul[ \L[aa] \L[bb] ] `` does not parse as: `` \Ul[\L[aa]<NEWLINE>\L[bb]<NEWLINE>] `` but rather as: `` \Ul[\L[aa]\L[bb]] `` because the <`content` argument> of `ul` is marked with `remove_whitespace_children` and automatically removes any whitespace children (such as a newline) as a result. This also applies to consecutive sequences of <auto parent> macros, e.g.: `` \L[aa] \L[bb] `` also does not include the newline between the list items. The definition of whitespace is the same as the ASCII whitespace definition of HTML5: ` \r\n\f\t`. = Pseudo-macro argument property {parent=Macro argument} This section documents ways to classify macro arguments that are analogous to <macro argument properties>, but which don't yet have clear and uniform programmatic effects and so are a bit more hand wavy for now. = `content` argument {c} {parent=Pseudo-macro argument property} The <`content` argument> of macros contains the "main content" of the macro, i.e. the textual content that will show the most proeminently once the macro is rendered. It is usually, but not always, the first <positional argument> of macros. We should probably make it into an official <macro argument property> at some point. In most cases, it is quite obvious which argument is the <`content` argument>, e.g.: * <`\i` macro>: in `\i[asdf qwer]` then `asdf qwer` is the <`content` argument> * <`\a` macro>: in `\a[https://example.com][example website]` then `example website` is the <`content` argument> Some macros however don't have a <`content` argument>, especially when they don't show any textual acontent as their primary rendered output, e.g.: * <`\Image` macro>: this macro has `title` byt not content, e.g. as in: `\Image[flower.jpg]{title=}`, since the primary content is the `Image` rather than any specific text Philosophically, the <`content` argument> of a macro is analogous to the `innerHTML` of an HTML tag, as opposed to attributes such as `href=` and so on. The difference is that in <OurBigBook Markup>, every macro argument can contain child elements, while in HTML only the `innerHTML`, but not the attributes, can. = `title` argument {c} {parent=Pseudo-macro argument property} The <`title` argument> is an argument that gets used in <automatic ID from title> calculation of macro IDs. The <`title` argument> currently appears as both <positional arguments> and <named arguments> Examples: * <headers>: `= My header` is equivalent to `\H[1][My header]`, and `My header` is the <`title` argument>, which is a <positional argument> in this case * <images>: in `\Image[flower.jpg]{title=My header}` and `My header` is the <`title` argument>, which is a <named argument> in this case = `description` argument {c} {parent=Pseudo-macro argument property} The <`description` argument> is similar to the <`title` argument> in that it adds information about some block such as an <image> or <code block>. The difference from the title is that it does not count toward <automatic ID from title> calculations. = OurBigBook Markup concepts {parent=OurBigBook Markup} These are shared concepts that are used across other sections. = `auto_parent` macro property {id=auto-parent} {parent=OurBigBook Markup concepts} Some sequences of macros such as `l` from <lists> and `tr` from <tables> automatically generate implicit parents, e.g.: `` \Ul[ \L[aa] \L[bb] ] `` parses exactly like: `` \L[aa] \L[bb] `` The children are always added as arguments of the <`content` argument> of the implicit parent. If present, the `auto_parent` macro property determines which auto-parent gets added to those macros. = Block vs inline macros {parent=OurBigBook Markup concepts} Every OurBigBook macro is either block or inline: * a block macro is one that takes up the entire line when rendered All block macros start with a capital letter, e.g. `\H` for <headers>. * and an inline macro is one that goes inside of a line. Every inline macro starts with a lowercase letter e.g. `\a` for <links>. Some macros have both a block and an inline version, and like any other macro, those are differentiated by capitalization: * <mathematics> * <code blocks> * <comments> = Block macro {parent=Block vs inline macros} See: <Block vs inline macros>{full}. = Inline macro {parent=Block vs inline macros} See: <Block vs inline macros>{full}. = Known URL protocols {parent=OurBigBook Markup concepts} Certain common URL protocols are treated as "known" by OurBigBook, and when found they have special effects in some parts of the conversion. The currently known protocols are: * `http://` * `https://` Effects of known protocols include: * <shorthand link parsing rules>: mark the start of shorthand links * <store images in a separate media repository>: mark an image `src` to ignore `provider` = JavaScript case conversion {c} {parent=OurBigBook Markup concepts} Some parts of OurBigBook use "JavaScript case conversion". This means that the conversion is done as if by the `toLowerCase`/`toUpperCase` functions. The most important fact about those functions is that they do convert non-ASCII Unicode capitalization, e.g. between `É` and `é`: * https://stackoverflow.com/questions/3590833/does-javascript-string-tolowercase-follow-unicode-standards-in-case-conversion * https://stackoverflow.com/questions/929079/unicode-lowercase-characters These conversions are also specified in the Unicode standard. = JavaScript regular expression {c} {parent=OurBigBook Markup concepts} https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions = Git tracked project {c} {parent=OurBigBook Markup concepts} If the <project toplevel directory> of an OurBigBook project is also a git repository, and if `git` is installed, then the OurBigBook project is said to be a "Git tracked project". = Element ID {c} {parent=OurBigBook Markup concepts} = ID {c} {synonym} In general usages of a <macro> produces an element, and every element has an ID. IDs must be unique, and they are used as the target of <internal links>. E.g. due to <automatic ID from title>{full}, the elements: `` = Animal == Big dog I like <big dogs>. `` would have IDs respectively: * `animal` * `big-dog` Such IDs are almost always rendered as HTML IDs as something like: `` <h1 id="animal"> <h2 id="big-dog"> `` and can therefore be linked to in a page with the corresponding fragment: `` animal.html#big-dog `` = Reserved ID {parent=element-id} = Reserved IDs {synonym} The following IDs are reserved: * `-`: reserved as a path component anywhere in an ID, in both static and web output. This forbids `-`, `-/child`, `parent/-`, and `parent/-/child`. * `_out`: forbidden as the first component of an ID, to protect local build output. * `.git`: forbidden as the first component of an ID, to protect Git metadata. * `index`: reserved to prevent collisions with the home page's `index.html`. This check also applies to headers inside scopes. Other uses of hyphens, such as `my-article`, and leading underscores, such as `_toc` and `_1`, are allowed. `_out` and `.git` are allowed in nested IDs such as `parent/_out` and `parent/.git`. OurBigBook uses the `-/` namespace for generated IDs. For example: * the <table of contents> uses the fixed ID `-/toc` * elements without an explicit ID may receive automatically generated IDs of type `-/1`, `-/2` and so on If you use a reserved ID, you will get an error message such as: `` error: tmp.bigb:3:1: The ID path component "-" is reserved for website routes: "-/toc" `` = OurBigBook CLI {c} {parent=} = `ourbigbook` executable {synonym} {title2} OurBigBook CLI is the executable program called `ourbigbook` which comes when you install `npm install ourbigbook`. It is the main command line utility of the <OurBigBook Project>. Its functionality will also be exposed on GUI <editor support> such as <Visual Studio Code> to make things nicer for non-technical users. The main functionalities of the executable are to: * convert <OurBigBook Markup> files to HTML files or other formats The HTML files can then be either viewd from your filesystem on a browser, or uploaded and hosted very cheaply or for free so that others can see it, e.g. on <GitHub Pages>. * <publish your content>, either to <OurBigBook Web> or as a <static website> Or if you are a programmer: OurBigBook CLI is a Static Wiki generator that can be invoked from the command line with the <ourbigbook executable>. OurBigBook CLI is how https://cirosantilli.com[] is published. OurBigBook Web takes as input the exact same format of OurBigBook Markup files used by OurBigBook CLI. TODO support/improve import/export to/from OurBigBook Web, see also: <--web>. The OurBigBook CLI calls the <OurBigBook Library> to convert each input file. Convert a `.bigb` file to HTML and output the HTML to a file with the same basename without extension, e.g.: `` ourbigbook hello.bigb firefox _out/html/hello.html `` Files named `index.bigb` are converted to `index.html` so that they will show on the website's base address: `` ourbigbook index.bigb firefox _out/html/index.html `` Convert all `.bigb` files in a directory to HTML files, e.g. `somefile.bigb` to `_out/html/somefile.html`: `` ourbigbook . `` The HTML output files are placed right next to each corresponding `.bigb`. The output file can be selected explicitly with: <outfile>. Output to stdout instead of saving it to a file: `` ourbigbook --stdout index.bigb `` In order to resolve <cross file internal links>, this actually does two passes: * first an <ID extraction> pass, which parses all inputs and dumps their IDs to the ID database * then a second render pass, which uses the IDs in the ID database = stdin conversion {parent=OurBigBook CLI} = stdin {synonym} Convert a `.bigb` file from stdin to HTML and output the contents of `<body>` to stdout: `` printf 'ab\ncd\n' | ourbigbook --body-only `` Stdin converion is a bit different from conversion from a file in that it ignores the <ourbigbook.json> and any other setting files present in the current directory or its ancestors. Also, it does not produce any changes to the <ID database>. In other words, a conversion from stdin is always treated as if it were outside of any project, and therefore should always produce the same results regardless of the current working directory. = OurBigBook CLI quick start {c} {parent=OurBigBook CLI} = Play with the template {parent=OurBigBook CLI quick start} Learn the syntax basics in 5 minutes: https://docs.ourbigbook.com/-/obb/dist/editor[]. First ensure that Node.js is installed in your computer. You should be able to successfully the following command successfully from a terminal: `` node --version `` Now let's play with https://github.com/ourbigbook/template[an OurBigBook template] locally: `` git clone https://github.com/ourbigbook/template cd template npm install npx ourbigbook . firefox _out/html/index.html `` That template can be seen rendered live at: http://cirosantilli.com/ourbigbook-generate-multifile/ Other templates are documented at: <--generate>. To <publish to GitHub Pages> on your repository you can just fork the repository https://github.com/ourbigbook/template to your own https://github.com/johndoe/template and then: `` git remote set-url origin git@github.com:johndoe/template.git npx ourbigbook --publish `` and it should now be visible at: https://johndoe.github.io/template Then, every time you make a change you can publish the new version with: `` git add . git commit --message 'hacked stuff' ourbigbook --publish . `` or equivalently with the <publish commit> shortcut: `` ourbigbook --publish-commit 'hacked stuff' `` If you want to publish to your root page https://johndoe.github.io instead of https://johndoe.github.io/template you need to rename the `master` branch to `dev` as mentioned at <publish to GitHub pages root page>: `` git remote set-url origin git@github.com:johndoe/johndoe.github.io.git # Rename master to dev, and delete the old master. git checkout -b dev git push origin dev:dev git branch -D master git push --delete origin master npx ourbigbook --publish `` The following files of the template control the global style of the output, and you are free to edit them: * `ourbigbook.liquid.html`: global HTML template in https://shopify.github.io/liquid/[Liquid format]. Available variables are documented at <--template>{full} * `main.scss`: https://sass-lang.com/[Sass] file that gets converted to raw CSS `main.css` by `npx ourbigbook .`. Sass is just much more convenient to write than raw CSS. That file gets included into the global HTML template inside `ourbigbook.liquid.html` at: `` <link rel="stylesheet" href="{{ root_relpath }}main.css"> `` = Important command line options {parent=OurBigBook CLI quick start} When you run: `` npx ourbigbook . `` it converts all files in the current directory separately, e.g.: * `index.bigb` to `_out/html/index.html`, since `index` is a magic name that we want to show on the root URL * `not-index.bigb` to `_out/html/not-index.html`, as this one is a regular name unlike `index` * `main.scss` to `main.css` If one of the input files starts getting too large, usually the toplevel `index.bigb` in which you dump everything by default like Ciro does, you can speed up development and just compile files individually with either: `` npx ourbigbook index.bigb npx ourbigbook not-index.bigb `` Note however that when those individual files have a <cross file internal link> to something defined in `not-index.bigb`, e.g. via `\x[h2-in-not-the-index]`, then you must have first previously done pass once with: `` npx ourbigbook . `` to parse all files and extract all necessary IDs to the <ID database>. That would be optimized slightly with the <--no-render> command line option: `` npx ourbigbook --no-render . `` to only extract the IDs but not render, which speeds things up considerably When dealing with large files, you might also be interested in the following amazing options: * <split headers> * <\H splitDefault argument> = Produce a standalone HTML file {parent=Important command line options} To produce a single standalone output file that contains everything the viewer needs to correctly see the page do: `` npx ourbigbook --embed-resources --embed-includes index.bigb `` You can now just give the generated `_out/html/index.html` to any reader and they should be able to view it offline without installing anything. The flags are: * <--embed-includes>: without this, `\Include[not-index]` shows as a link to the file `_out/html/not-index.html` which comes from `not-index.bigb` With the flag, `not-index.bigb` output gets embedded into the output `_out/html/index.html` directly * <--embed-resources>: by default, we link to CSS and JavaScript that lives inside `node_modules`. With this flag, that CSS and JavaScript is copied inline into the document instead. One day we will try to handle <images> that way as well = Useless knowledge {parent=OurBigBook CLI quick start} Install the NPM package globally and use it from the command line for a quick conversion: `` npm install -g ourbigbook printf 'ab\ncd\n' | ourbigbook --body-only `` or to a file: `` printf 'ab\ncd\n' | ourbigbook > tmp.html `` You almost never want to do this except when <developing OurBigBook>, as it won't be clear what version of `ourbigbook` the document should be compiled with. Just be a good infant and use OurBigBook <play with the template>[with the template] that contains a `package.json` via `npx`, OK? Furthermore, the default install of Chromium on Ubuntu 21.04 uses Snap and blocks access to dotfiles. For example, in a sane NVM install, our global CSS would live under `/home/ciro/.nvm/versions/node/v14.17.0/lib/node_modules/ourbigbook/-/obb/ourbigbook.css`, which gets blocked because of the `.nvm` part: * https://forum.snapcraft.io/t/dot-files/7062 * https://bugs.launchpad.net/snapd/+bug/1607067 * https://superuser.com/questions/1546550/chromium-81-wont-display-dotfiles-anymore * https://askubuntu.com/questions/1184357/why-cant-chromium-suddenly-access-any-partition-except-for-home * https://askubuntu.com/questions/1214346/as-a-user-is-there-any-way-to-change-the-confinement-of-a-snap-package One workaround is to use <--embed-resources>, but this of course generates larger outputs. To run master globally from source for development see: <run OurBigBook master>{full}. This one actually works despite the dotfile thing since your development path is normally outside of dotfiles. Try out the JavaScript API with \a[lib_hello.js]: `` npm install ourbigbook ./lib_hello.js `` = Publish your content {parent=OurBigBook CLI} = Publish {synonym} = Publishing {synonym} = Published {synonym} There are two ways to publish your <OurBigBook> content: * as a <static website>. This means that you generate HTML files from <OurBigBook Markup> files and then publish them either by: * uploading to a static website server such as <GitHub Pages> by using the <--publish> option * converting with <--publish-target `local`> and sending a zip with the pages to someone to view locally * to an <OurBigBook Web> instance such as <OurBigBook.com>. This can be done either by: * editing on the <OurBigBook Web editor> directly in your browser * uploading <OurBigBook Markup> files from your computer with the <--web> option A fundamental design choice of the <OurBigBook Project> is that, except for bugs, a single <OurBigBook Markup> source tree can be published in both of those ways without any changes. The trade-offs between the two options are highlighted at: <OurBigBook Web vs static website publishing>. \Image[feature/local-editing/bigb-publish-to-web-or-static-editor-logos.svg] {title=You can <publish> local lightweight markup files to either <OurBigBook Web> or as a <static website>} {provider=github} {height=600} \Video[https://www.youtube.com/watch?v=Ghvlztiu6rI] {title=Edit locally and publish demo} {height=720} {width=720} = OurBigBook Web vs static website publishing {parent=Publish your content} * <static websites> are cheaper to host, including many free options such as <GitHub Pages>. This means that you are likely to always have several free or cheap choices of where to upload your content to, making it essentially all but https://en.wiktionary.org/wiki/TEOTWAWKI[TEOTWAWKI]-proof Pages will also load slightly fatser. * <OurBigBook Web> has killer multi-user features: <OurBigBook Web Topics>, article upvotes and <OurBigBook Web discussions> Furthermore, it also has some non multi-user features which cannot be feasibly implemented in a static website because they would require too much storage, on the fly generation is the only feasible way to deal with them: * <OurBigBook Web dynamic article tree> * article history. Unimplemented as of writing: https://github.com/ourbigbook/ourbigbook/issues/248 Its main downside is that it is more expensive to host. The <OurBigBook Project> will do its best to keep <OurBigBook.com> uploading as free as possible, but upload limits necessarily have to be more strict than those of static websites, as the underlying operating cost is larger. = Index file {parent=OurBigBook CLI} An index file is a file with basename `index.bigb`. Those basenames have the following magic properties: * the default output file name for an index file in HTML output is either: * `index.html` when in the <project toplevel directory>. E.g. `index.bigb` renders to `index.html`. Note that GitHub and many other static website hosts then automatically hide the `index.html` part from the URL, so that your `index.bigb` hosted at `http://example.com` will be accessible simply under `http://example.com` and not `http://example.com/index.html` * the name of the subdirectory in which it is located when not in the <project toplevel directory>. E.g. `mysubdir/index.bigb` outputs to `mysubdir.html` Previously, we had placed the output in `mysubdir/index.html`, but this is not as nice as it makes GitHub pages produce URLs with a trailing slash as `mysubdir/`, which is ugly, see also: https://stackoverflow.com/questions/5948659/when-should-i-use-a-trailing-slash-in-my-url * the default <the toplevel header>[toplevel header] ID of an index files is derived from the parent directory basename rather than from the source file basename = Project toplevel directory {parent=Index file} This directory is determined by first checking the presence of a <ourbigbook.json> file. If a <ourbigbook.json> is found, then the project toplevel directory is the directory that contains that file. * otherwise, if the input path is a descendant of the current working directory, then the current working directory is used, see also: <the current working directory does not matter when there is a ourbigbook.json> * otherwise, if the input path is a directory, it is used * otherwise, the directory containing the input file is used For example, consider the file following file structure relative to the current working directory: `` path/to/notindex.bigb `` In this case: * if there is no `ourbigbook.json` file: * if we run `ourbigbook .`: the toplevel directory is the current directory `.`, and so `notindex.bigb` has ID `path/to/notindex` * if we run `ourbigbook path`: same * if we run `ourbigbook path/to`: same * if we run `ourbigbook path/to/notindex.bigb`: same * if there is a `path/ourbigbook.json` file: * if we run `ourbigbook .`: the toplevel directory is the current directory `.` because the `ourbigbook.json` is below the entry point and is not seen, and so `notindex.bigb` has ID `path/to/notindex` * if we run `ourbigbook path`: the toplevel directory is the directory with the `ourbigbook.json`, `path`, and so `notindex.bigb` has ID `to/notindex` * if we run `ourbigbook path/to`: same * if we run `ourbigbook path/to/notindex.bigb`: same = The toplevel index file {parent=Project toplevel directory} This is the <index file> present in the <project toplevel directory>, i.e. the file with path exactly `index.bigb`. = The home article {parent=The toplevel index file} {tag=Toplevel header} = The index article {synonym} The "home article" is the first article of <the toplevel index file>. E.g. in: index.bigb `` = John Smith's Homepage == I like dogs `` then "John Smith's Homepage" is the home article, but "I like dogs" is not. The home article has some special handling done to it, notably: * it renders as "Home" in many places as such as the breadcrumb, a way to make things more unified and succinct, especially on web * it automatically gets two IDs: * a main empty ID * a <`\H` synonym argument>[synonym] to that empty ID For example we could write: `` = John Smith's Homepage == I like dogs This is my homepage: <> And also: <John Smith's Homepage> `` Here * `<>` renders as "Home" * `<John Smith's Homepage>` renders as `John Smith's Homepage` but both link to the same location As a consequence of the ID being empty, you have to set <H parent argument> of subsequent headers to empty if you witsh to use them as in: `` = John Smith's Homepage = I like dogs {parent=} `` Doing: `` = John Smith's Homepage = I like dogs {parent=John Smith's Homepage} `` doesn't current work because the ID `john-smith-s-homepage` is just a synonym to the empty ID, and you can't currently set `parent=` to point to synonyms: This is because the ID John Smith's Homepage You can prevent a second ID from being giving by simply setting the ID to be explicitly empty: `` = John Smith's Homepage {id=} `` which would generte just a single empty ID. = The current working directory does not matter when there is a `ourbigbook.json` {parent=The toplevel index file} When the file or directory being converted has an ancestor directory with a `ourbigbook.json` file, then your current working directory does not have any effect on OurBigBook output. For example if we have: `` /project/ourbigbook.json /project/index.bigb /project/subdir/index.bigb `` then all of the following conversions produce the same output: * directory conversion: * `cd /project && ourbigbook .` * `cd / && ourbigbook project` * `cd project/subdir && ourbigbook ..` * file conversion: * `cd /project && ourbigbook index.bigb` * `cd / && ourbigbook project/index.bigb` * `cd project/subdir && ourbigbook ../index.bigb` When there isn't a `ourbigbook.json`, everything happens as though there were an empty `ourbigbook.json` file in the current working directory. So for example: * outputs that would be placed relative to inputs are still placed in that place, e.g. `index.bigb -> index.html` always stay together * outputs that would be placed next to the `ourbigbook.json` are put in the current working directory, e.g. <the out directory> Internally, the general philosophy is that the JavaScript API in \a[index.js] works exclusively with paths relative to the <project toplevel directory>. It is then up to callers such as \a[ourbigbook] to ensure that filesystem specifics handle the relative paths correctly. = OurBigBook CLI options {c} {parent=OurBigBook CLI} = `ourbigbook` executable options {synonym} = `--check-db-only` {parent=OurBigBook CLI options} Check the database for consistency, e.g. duplicated IDs. Don't do anything else, including <ID extraction>, which must have been done previously. The initial use case was for usage in <parallel builds>. = `--china` {parent=OurBigBook CLI options} This is the most important option of the software. It produces a copy of the HTML of https://cirosantilli.com/china-dictatorship to stdout. The data is stored inside an NPM package, making it hard to censor that information, see also: https://cirosantilli.com/china-dictatorship#mirrors Usage: `` ourbigbook --china > china.html firefox china.html `` = `--dry-run` {parent=OurBigBook CLI options} The `--dry-run` option is a good way to debug the <--publish option>, as it builds the publish output files without doing any git commands that would be annoying to revert. So after doing: `` ourbigbook --dry-run --publish . `` you can just go and inspect the generated HTML to see what would get pushed at: `` cd _out/publish/_out/publish/ `` see also: <the out directory>. Inspiration: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/6d0a900f4c3c15e65d850f9d29d63315a6f976bf#dry-run-to-get-commands-for-your-project = `--dry-run-push` {parent=--dry-run} Similar to <--dry-run>, but it runs all git commands except for `git push`, which gives a clearer idea of what `--publish` would actually do including the git operations, but without publishing anything: `` ./ourbigbook --dry-run --publish . `` = `--embed-includes` {parent=OurBigBook CLI options} Makes <includes> render the included content in the same output file as the include is located, instead of the default behaviour of creating links. For example given: index.bigb `` = Index \Include[notindex] `` notindex.bigb `` = Notindex A paragraph in notindex. == Notindex 2 `` then for conversion with: `` ourbigbook --embed-includes index.bigb `` then the output `index.html` contains an output equivalent to if your input file were: `` = Index == Notindex A paragraph in notindex. === Notindex 2 `` Note that a prior <ID extraction> pass is not required, `--embed-includes` just makes `\Include` read files as they are found in the source. In addition to this: * <cross file internal links> outside the included files are disabled, and the cross file ID database does not get updated. It should be possible to work around this, but we are starting with the simplest implementation that forbids it. TODO at: https://github.com/ourbigbook/ourbigbook/issues/343 The problem those cause is that the IDs of included headers show as duplicate IDs of those in the ID database. This should be OK to start with because the more common use case with `--html-single-page` is that of including all headers in a single document. TODO: this option is gone. Otherwise, `include` only adds the headers of the other file to the table of contents of the current one, but not the body of the other file. The ToC entries then point to the headers of the included external files. You may want to use this option together with <--embed-resources> to produce fully self-contained individual HTML files for your project. = `--escape-literal` {parent=OurBigBook CLI options} Take input from stdin as a plaintext string and produce output escaping the input string so that OurBigBook conversion will result in a single output string. E.g.: `` printf '\\a[`\n' | ourbigbook --escape-literal `` outputs: `` \\a\[\`\ `` so that every <escape characters>[character that needed to be escaped] was fully escaped to produce plaintext output. Note that newlines are also currently escaped even when that is not strictly needed, e.g. in the case of double newlines it would be needed but not for single newlines. But we are keeping it simple for now. = `--embed-resources` {parent=OurBigBook CLI options} Embed as many external resources such as images and CSS as possible into the HTML output files, rather than linking to external resources. For example, when converting a simple document to HTML: index.bigb `` = Index My paragraph. `` with: `` ourbigbook index.bigb `` the output contains references to where OurBigBook is installed in our local filesystem: `` <style> @import "/home/ciro/bak/git/ourbigbook/-/obb/ourbigbook.css"; </style> <script src="/home/ciro/bak/git/ourbigbook/-/obb/ourbigbook_runtime.js"></script> `` The advantage of this is that we don't have to duplicate this for every single file. But if you are giving this file to someone else, they would likely not have those files at those exact locations, which would break the HTML page. With `--embed-resources`, the output contains instead something like: `` <style>/*! normalize.css v8.0.1 | MIT License | github.com/necolas/normalize.css */html{ [[ ... A LOT MORE CSS ... ]]</style> <script>/*! For license information please see ourbigbook_runtime.js.LICENSE.txt */ !function(t,e){"object"==typeof exports&&"object"==typeof module?module.exports=e() [[ ... A LOT MORE JAVASCRIPT ... ]]</script> `` This way, all the required CSS and JavaScript will be present in the HTML file itself, and so readers will be able to view the file correctly without needing to install any missing dependencies. The use case for this option is to produce a single HTML file for an entire build that is fully self contained, and can therefore be given to consumers and viewed offline, much like a PDF. Examples of embeddings done: * CSS and JavaScript are copy pasted in place into the HTML. The default built-in CSS and JavaScript files used by OurBigBook (e.g. the KaTeX CSS <mathematics>[used for mathematics]) are currently all automatically downloaded as NPM package dependencies to ourbigbook Without `--embed-resources`, those CSS and JavaScript use their main cloud CDN URLs, and therefore require Internet connection to view the generated documents. The embedded version of the document can be viewed offline however. There is however a known bug: KaTeX fonts are not currently embedded, so math won't work properly. The situation is similar as for images, but a bit harder because we also need to fetch the blobs from the CSS, which is likely doable from Webpack: * https://github.com/ourbigbook/ourbigbook/issues/157 * https://stackoverflow.com/questions/41657087/webpack-inline-font-with-url-loader * https://stackoverflow.com/questions/35369419/how-to-use-images-in-css-with-webpack Examples of embedding that could be implemented in the future: * <images> are downloaded if needed and embedded as `data:` URLs. Doing this however has a downside: it would slow the page loading down. The root problem is that HTML was not designed to contain assets, and notably it doesn't have byte position indices that can tell it to skip blobs while parsing, and how to refer to them later on when they show up on the screen. This is kind of why EPUB exists: https://github.com/ourbigbook/ourbigbook/issues/158 Images that are managed by the project itself and already locally present, such as those inside the project itself or due to <ourbigbook.json/media-providers> usually don't require download. For images linked directly from the web, we maintain a local download cache, and skip downloads if the image is already in the cache. To re-download due to image updates, use either: * `--asset-cache-update`: download all images such that the local disk timestamp is older than the HTTP modification date with https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since[`If-Modified-Since`] * `--asset-cache-update-force`: forcefully redownload all assets Keep in mind that certain things can never be embedded, e.g.: * YouTube videos, since YouTube does not offer any download API = -`F`, `--force-render` {id=force-render} {parent=OurBigBook CLI options} Always render all selected files, irrespectively of if they are known to be outdated or not. OurBigBook stores the timestamp of the last successful <ID extraction> step for each file. For <ID extraction>, we always skip the extraction if the filesystem timestamp of a source file is older than the last successful extraction. For render: * we mark output files as outdated when the corresponding source file is parsed * we also skip rendering non-outdated files by default when you invoke ourbigbook on a directory, e.g. `ourbigbook .`, as this greatly speeds up the interactive error fixing turnaround time * we always re-render fully when you specify a single file, e.g. `ourbigbook path/to/index.bigb` However, note that skipping renders, unlike for ID extraction, can lead to some outdated pages. This option disables the timestamp skip for rendering, so ensure that you will get a fully clean updated render. E.g. consider if you had two files: file1.bigb `` = File 1 == File 1 1 `` file2.bigb `` = File 2 == File 2 1 \x[file-1-1] `` We then do the initial conversion: `` ourbigbook . `` we see output like: `` extract_ids file1.bigb extract_ids file1.bigb finished in 45.61287499964237 ms extract_ids file2.bigb extract_ids file2.bigb finished in 15.163879998028278 ms render file1.bigb render file1.bigb finished in 23.21016100049019 ms render file2.bigb render file2.bigb finished in 25.92908499762416 ms `` indicating full conversion without skips. But then if we just modify fil1.bigb as: `` = File 1 == File 1 1 hacked {id=file-1-1} `` the following conversion with `ourbigbook .` would look like: `` extract_ids file1.bigb extract_ids file1.bigb finished in 45.61287499964237 ms extract_ids file2.bigb extract_ids file2.bigb skipped by timestamp render file1.bigb render file1.bigb finished in 41.026930000633 ms render file2.bigb render file2.bigb skipped by timestamp `` and because we skipped `file2.bigb` render, it will still have the outdated "File 1 1" instead of "File 1 1 hacked". We could in principle solve this problem by figuring out exactly which files need to be changed when a given ID changes, and we already <conversion process overview>[have to solve a similar problem due to query bundling]. Also, this will need to be done sonner or later for the <OurBigBook Web>. But lazy now: https://github.com/ourbigbook/ourbigbook/issues/207[], this is hard stuff. = `--format-source` {parent=OurBigBook CLI options} Parse and overwrite the local .bigb <OurBigBook Markup> input source files with the recommended code format. E.g.: `` ourbigbook index.bigb `` overwrites `index.bigb` with the recommended formatting, and: `` ourbigbook . `` does that for every single file in the current directory. This option uses the <bigb output format>. In order to reach a final stable state, you might need to run the conversion twice. This is not ideal but we don't have the patience to fix it. The reason is that links in image titles may expand twice. This is the usual type of two level recursion that has caused much more serious problems, see e.g. <\x within title restrictions>. E.g. starting with: `` <image my big dog> \Image[image.png]{title=My <big dog>} = Big dog `` the first conversion leads to uppercasing inside the image title: `` <image my big dog> \Image[image.png]{title=My <big Dog>} = Big Dog `` and the second one to uppercasing the reference to the image title: `` <image my big Dog> \Image[image.png]{title=My <big Dog>} = Big Dog `` = `--generate` {parent=OurBigBook CLI options} = Project template {synonym} = Template project {synonym} = Template repo {synonym} The project templates are simple ourbigbook project directories that serve as good starting point for new ourbigbook projects. They also contain useful examples of <OurBigBook Markup> usage to help users get started quickly. Generate one of the template repositories locally: * `ourbigbook --generate default`: a good starter template that illustrates many key OurBigBook features * https://github.com/ourbigbook/template * https://ourbigbook.github.io/template * `ourbigbook --generate min`: a minimal template that is still sane * https://github.com/ourbigbook/template-min * https://ourbigbook.github.io/template-min * `ourbigbook --generate subdir`: a template in which OurBigBook source is located a subdirectory `docs/`: * https://github.com/ourbigbook/template-subdir * https://ourbigbook.github.io/template-subdir This template illustrates that everything works exactly as if OurBigBook source were in the git repository toplevel. This is a convenient setup for programming projects that want to use OurBigBook for their documentation without polluting their toplevel. End users almost never want this, because it means that to have a sane setup you need to: * install OurBigBook globally with `npm install -g ourbigbook` * generate the template * then install OurBigBook locally again with `npm install` so maybe we should just get rid of that option and just ensure that we can provide an up-to-date working template for the latest relase. For now we are keeping this as it is useful to automate the updating of templates during the <release procedure>. = `--help-macros` {parent=OurBigBook CLI options} You can get an overview of all macros in JSON format with: `` ourbigbook --help-macros `` = `--log` {parent=OurBigBook CLI options} Give multiple times to enable a list of certain types of logs to stderr help debugging, e.g.: `` ./ourbigbook --log ast tokens -- index.bigb `` Note that this follows https://github.com/tj/commander.js/tree/e0e723810357e915210af38ccf5098ffe1fb8e65#variadic-option[commander.js' shorthand variadic argumentso syntax], and thus the `--` is required above. If you want to omit it for a single value you have to add the `=` sign as in: `` ./ourbigbook --log=ast index.bigb `` Values not documented in other sections: * `ast`: the full final parsed <abstract syntax tree> as JSON * `ast-simple`: a simplified view of the <abstract syntax tree> with one AstNode or AstArgument per line and showing only the most important fields * `ast-pp-simple`: view snapshots of the various <abstract syntax tree> post process stages, more info at: <conversion process overview> * `ast-inside`: print the AST from inside the `ourbigbook.convert` call before it returns. This is useful to debug the program if `ourbigbook.convert` blows up on the next stages before returning. * `db`: show database transactions done by OurBigBook, to help debug stuff like <cross file internal links> * `mem`: show process memory usage as per Node.js' `process.memoryUsage()` after each <--log perf> step: https://stackoverflow.com/questions/12023359/what-do-the-return-values-of-node-js-process-memoryusage-stand-for[]. Implies <--log perf>. To use this options, you must run OurBigBook with the `--expose-gc` command line option, e.g. with: `` node --expose-gc $(which ourbigbook) myfile.bigb `` * `parse`: parsing steps * `split-headers`: log which split header is currently being rendered. This can be extremelly helpful to identify which logs that we've added that come from the header of interest. * `tokenize`: tokenization steps * `tokens`: final parsed token stream * `tokens-inside`: like `ast-inside` but for tokens. Also adds token index to the output, which makes debugging the parser way easier. = `--log headers` {parent=--log} This nifty little option outputs to stderr what the header graph looks like! It is a bit like a <table of contents> in your terminal, for when you need to have a look at the outline of the document to decide where to place a new header, but are not in the mood to open a browser or use the <browser editor with preview>. Sample output excerpt for this document: `` = h1 ourbigbook == h2 1 quick-start == h2 2 design-goals === h3 2.1 saner === h3 2.2 more-powerful == h2 3 paragraphs == h2 4 links `` This option can also serve as a debug tool for header tree related features (confession: that was its original motivation!). TODO = `--log perf` {parent=--log} print <performance> statistics to stderr, for example `` ./ourbigbook --log=perf index.bigb `` could output: `` perf start: 181.33060800284147 perf tokenize_pre: 181.4424349963665 perf tokenize_post: 318.333980999887 perf parse_start: 319.1866770014167 perf post_process_start: 353.5477180033922 perf post_process_end: 514.1527540013194 perf render_pre: 514.1708239987493 perf render_post: 562.834307000041 perf end: 564.0349840000272 perf convert_input_end 566.1234430000186 perf convert_path_pre_sqlite 566.1564619988203 perf convert_path_pre_sqlite_transaction 566.2528780028224 perf convert_path_post_sqlite_transaction 582.256645001471 perf convert_path_end 582.3469280004501 `` which shows how long different parts of the <conversion process overview>[conversion process] took to help identify bottlenecks. This option can also be useful to mark phases of the conversion to identify from which phase other logs are coming from, e.g. if we wanted to know which part of the conversion is making a ton of database requests we could run: `` ourbigbook --log db perf -- index.bigb `` and we would see the database requests made at each conversion phase. Note that `--log perf` currently does not take sub-converts into account, e.g. <include> and <\OurBigBookExample> both call the toplevel conversion function `convert`, and therefore go through all the conversion intervals, but we do not take those it account, and just dump them all into the same toplevel interval that they happen in, currently between `post_process_start` and `post_process_end`. = `--no-check-db` {parent=OurBigBook CLI options} = `--skip-check-db` {synonym} Skip the database sanity check that is normally done after the <ID extraction> step. This was originally added to speed up, originally added to speed up the <web upload> development loop, when we knew that there were no errors in the database after a local conversion, and wanted to get to the upload phase faster, but the DB check can take several seconds for a large input. But it then later also found usage with <parallel builds> followed by a <--check-db-only>. = `--no-db` {parent=OurBigBook CLI options} Don't use the <ID database> during this run. This implies that the on-disk database is not read, and also not written to. Instead, a temporary clean in-memory database is used. = `--no-html-x-extension` {parent=OurBigBook CLI options} If not given, <internal links> render with the `.html` extension as in: `` <a href=not-index.html#h2-in-not-the-index> `` This way, those links will work when rendering locally to `.html` files which is the default behaviour of: `` ourbigbook . `` If given however, the links render without the `.html` as in: `` <a href=not-index#h2-in-not-the-index> `` which is what is needed for servers such as GitHub Pages, which automatically remove the `.html` extension from paths. This option is automatically implied when <Publish to GitHub Pages>[publishing to targets that remove the `.html` extension such as GitHub pages]. = `--no-render` {parent=OurBigBook CLI options} Only extract IDs to fill the <ID database>, don't render. This saves time if you only want to render a single file which has references to other files without getting any errors. = `--no-web-render` {parent=OurBigBook CLI options} Same as <`--no-render`>, but for the <--web> upload stage. Web upload consists of two stages: * extract local ids and render to split ourbigbook files. This can be disabled with <--no-render> * upload to web first on an <ID extraction> pass, and then a render pass. <`--no-web-render`> skips that render pass = `--outdir <outdir>` {id=outdir} {parent=OurBigBook CLI options} Set a custom output directory for the conversion. If not given, the <project toplevel directory> is used. Suppose we have an input file `./test.bigb`. Then: `` ourbigbook --outdir my_outdir test.bigb `` places its output at: `` my_outdir/test.html `` The same would happen if we instead did a full directory conversion as in: `` ourbigbook --outdir my_outdir . `` The output would also be placed in `my_outdir/test.html`. This option also relocates <the out directory> to the target destination, e.g.: `` ourbigbook --outdir my_outdir test.bigb `` would generate: `` my_outdir/out `` This means that the source tree remains completely clean, and every output and temporary cache is put strictly under the selected `--outdir`. = `--outfile <outfie>` {id=outfile} {parent=OurBigBook CLI options} Save the output to a given file instead of outputting to stdout: `` ./ourbigbook --outfile not-index.html not-index.bigb `` The generated output is slightly different than that of: `` ./ourbigbook not-index.bigb > not-index.html `` because with `--outfile` we know where the output is going, and so we can generate relative includes to default CSS/JavaScript files. = `-I --input-format <inputformat>` {id=input-format} {parent=OurBigBook CLI options} Selects the input format. Supported values: * `bigb`: the default an recommended format * `md`, see: <--input-format md> The input format is automatically selected from the file extension, e.g. `.md` or `.markdown` input files. For input from standard input, it can be selected explicitly e.g. with: `` printf '# Hello' | ourbigbook -I markdown -O bigb --stdout `` Directory conversion remains BigB-only unless `-I markdown` is given, so that Markdown documentation files such as `README.md` are not compiled unexpectedly. = `--input-format md` {parent=input-format} = Markdown input {synonym} {title2} OurBigBook has basic support for markdown input. Markdown is a more insane format and for now has no OurBigBook-specific features, but Codex kind of vibe coded it quickly and we did it for the lols. For now, you can transparently <include> standard markdown files such as \a[not-index-md.md] with: `` \Include[not-index-md] `` and OurBigBook will integrate it into the current project seamlessly. Live demo: \Include[not-index-md] = `-j`, `--jobs <n>` {id=jobs} {parent=OurBigBook CLI options} = Parallel build {c} {synonym} {title2} Directory builds use the available CPUs by default: `` ourbigbook . `` Use `-j N` or `--jobs N` to limit the number of worker processes. For example, `ourbigbook -j 4 .` uses up to four workers, and `ourbigbook -j 1 .` builds serially. The default is the number of CPUs available to the process, respecting CPU affinity. Workers are started only for files that need conversion and reused throughout the build. There are two parallel conversion stages, separated by a database check: * extract IDs from source files in parallel * wait for all extraction to finish, then run the <--check-db-only>[database consistency check] once, serially * render in parallel using the checked IDs and references Extraction or database errors stop the build before rendering. <--no-render> runs only extraction and the database check. Timestamp-based skipping still applies in both stages. SQLite uses write-ahead logging so readers can run while a writer commits. Only one worker at a time may update the ID database; the whole per-file transaction holds that write lock. Workers use fresh reference caches for each conversion. Generated file previews also render in parallel, before authored pages so explicit file headers retain precedence. Asset copying, Sass compilation, and directory listings remain serial. Web builds also extract IDs and render local split files in parallel, using the same `-j` setting. Updates to the local Web article cache are serialized. Uploads and server-side `web_render` requests remain serial. Single-file conversions, watch mode, source formatting, embedded includes, and builds without a persistent database retain their serial behavior. = `-O --output-format <outformat>` {id=output-format} {parent=OurBigBook CLI options} Default: <html output format>. = `bigb` output format {parent=output format} Outputs as <OurBigBook Markup>, i.e. the same format as the input itself! While using `-O bigb` is not a common use case, the existence of this format has the following applications: * automatic source code formatting e.g. with <--format-source>. The recommended format, including several edge cases, can be seen in the test file \a[test_bigb_output.bigb], which should be left unchanged by a `bigb` conversion. * manipulating source code on <OurBigBook Web> to allow editing either individual sections separatelly, or multiple sections at once * this could be adapted to allows us to migrate updates with breaking changes to the source code more easily. Alternatively on <OurBigBook Web>, we might just start storing the <AST> instead of source, and just rendering the source whenever users want to edit it. Can be tested interactively with: `` ourbigbook --no-db -O bigb --stdout --log=ast-simple test_bigb_output.bigb `` One important property of the `bigb` conversion is that is must not alter the <AST>, and therefore neither the final output, in any way. One good test is: `` ourbigbook index.bigb && mv _out/html/index.html _out/html/old.html && ourbigbook --format-source index.bigb && ourbigbook index.bigb && diff -u _out/html/old.html _out/html/index.html `` This was tracked at: https://github.com/ourbigbook/ourbigbook/issues/83 = `html` output format {parent=output format} The default output format. Web pages!!! = `md` output format {parent=output format} Outputs Markdown to a `.md` file. The portable common subset of OurBigBook headers, paragraphs, emphasis, links, images, code, quotes, lists, horizontal rules, mathematics and tables is converted to the closest Markdown construct. For example: `` ourbigbook -O md index.bigb `` Markdown output is generally viewed as a downgrade of data structures, since markdown is a subset of <OurBigBook>. One major use case is to publish directly GitHub, thus treating the markdown as an output entirely. = `adoc` output format {parent=output format} Outputs AsciiDoc to an `.adoc` file. It supports the same portable common subset as the <md output format>. For example: `` ourbigbook -O adoc index.bigb `` = `id` output format {parent=output format} This output format is used an intermediate step in <automatic ID from title>, that unlike the regular HTML output does not have any tags. It does not have serious applications to end users. We decided to expose it from the CLI mostly for fun, as it posed no extra work at all as it is treated internally exactly like any other conversion format. The <`id` output format> conversion is very simplistic: it basically just extracts the <`content` argument> of most macros. An important exception to that behaviour is the first argument of the <`\x` macro>: see <\x id output format>. For example, converting: `` \i[asdf] `` with the `id` output format produces simply: `` asdf `` instead of the HTML output: `` <i>asdf</i> `` This conversion type is useful in situations that users don't expect conversion to produce any HTML tags. For example, you could create a header: `` = My \i[asdf] `` and then following the <automatic ID from title> algorithm, that header would have the more commonly desired ID `my-asdf`, and not `my-<i>asdf</i>` or `my-i-asdf-i`. Similarly, any <macro argument> that references an ID undergoes <`id` output format> conversion. E.g. the above header could be referenced by: `` <My \i[asdf]> `` which is equivalent to: `` \x[my-asdf] `` Besides being more intuitive, this conversion also guarantees greater format portability, in case we ever decide to support other output formats besides HTML! <Macros> that don't have a <`content` argument> are just completely removed, i.e. typically non-textual macros such as <images>. We could put effort in outputting their title argument correctly, but meh, not worth the effort. The <`id` output format> also serves as a good start generalizing OurBigBook to multiple outputs, as this is a simple format. = `\x` `id` output format {parent=id output format} `\x` uses `href` if the content is not given explicitly. Previously, if `\x` didn't have a content, we were actually rendering the `\x` to calculate the ID. But then we noticed that doing so would require another parse pass, so we just went for this simpler approach. This is closely linked to <\x within title restrictions>. For example in: `` = Animal \x[image-i-like-dog] \Image[dog.jpg] {title=I \i[like] \x[dog]} == Dog hacked {id=dog} `` If you wanted `image-i-like-dog-hacked` instead, you would need to explicitly give it as in: `` = Animal \x[image-i-like-dog-hacked] \Image[dog.jpg] {title=I like \x[dog][dog hacked]} == Dog hacked {id=dog} `` For similar reasons as the above, `{p}` inflection with the <\x p argument> is not considered either, e.g. you would have: `` = Animal \x[image-i-like-dog] \Image[dog.jpg] {title=I like \x[dog]{p}} == Dog `` and not: `` \x[image-i-like-dogs] `` This can however be worked around with the <\x magic argument> as in; `` = Animal \x[image-i-like-dogs] \Image[dog.jpg] {title=I like <dogs>} == Dog `` = Unimplemented output formats {parent=id output format} = `latex` output format {parent=Unimplemented output formats} TODO: https://github.com/ourbigbook/ourbigbook/issues/38 One day, one day. Maybe. = `-p`, `--publish` {parent=OurBigBook CLI options} {tag=Publish your content} = `--publish` option {synonym} = Static website {synonym} = Static {synonym} = `ourbigbook --publish` {synonym} OurBigBook tooling is so amazing that we also take care of the HTML publishing for you! Once a publish target is properly setup, all you have to do is run: `` git add index.bigb git commit -m 'more content!' ourbigbook --publish `` and your changes will be published to the default target specified in <ourbigbook.json>. If not specified, e.g. with the the <--publish-target> option, the default target is to <publish to GitHub Pages>. Only changes committed to Git are pushed. Files that `ourbigbook` knows how to process get processed and only their outputs are added to the published repo, those file types are: * `.bigb` files are converted to `.html` * `.scss` files are converted to `.css` Every other Git-tracked file is pushed as is. When `--publish` is given, stdin input is not accepted, and so the current directory is built by default, i.e. the following two are equivalent: `` ./ourbigbook --publish ./ourbigbook --publish . `` Publishing only happens if the build has no errors. = `-P, --publish-commit <commit-message>` {id=publish-commit} {parent=OurBigBook CLI options} Like the <--publish option>, but also automatically: * `git add -u` to automatically add change to any files that have been previously git tracked * `git commit -m <commit-message>` to create a new commit with those changes This allows you to publish your changes live in a single command such as: `` ourbigbook --publish-commit 'my amazing change' . `` With great power comes great responsibility of course, but who cares! = `--publish-no-convert` {parent=OurBigBook CLI options} Attempt to publish without converting first. Implies the <--publish option>. This can only work if there was previously a successful publish conversion done, which later failed to publish during the following steps, e.g. due to a network error. This option was introduced for debugging purposes to help get the git commands right for large conversions that took a look time. = `--publish-target` {parent=OurBigBook CLI options} {tag=Publish your content} What type of target to publish for. The generated output of each publish target is stored under: `` _out/publish/_out/<target> `` e.g.: `` _out/publish/_out/local `` = `--publish-target github-pages` {parent=`--publish-target`} = Publish to GitHub Pages {synonym} {title2} = GitHub Pages {synonym} = Published to GitHub Pages {synonym} Publish to <#GitHub Pages>. https://pages.github.com/[GitHub pages] is the default OurBigBook publish target. Since that procedure is so important, it is documented directly at: <play with the template>. = Publish to GitHub pages root page {parent=publish-target-github-pages} If you want to publish your root user page, which appears at `/` (e.g. https://github.com/cirosantilli/cirosantilli.github.io for the user `cirosantilli`), GitHub annoyingly forces you to use the `master` branch for the HTML output: * https://github.com/isaacs/github/issues/212 * https://stackoverflow.com/questions/31439951/how-can-i-use-a-branch-other-than-master-for-user-github-pages This means that you must place your `.bigb` input files in a branch other than `master` to clear up `master` for the generated HTML. `ourbigbook` automatically detects if your repository is a root repository or not by parsing `git remote` output, but you must setup the branches correctly yourself. So on a new repository, you must https://stackoverflow.com/questions/42871542/how-to-create-a-git-repository-with-the-default-branch-name-other-than-master[first checkout to a different branch] as in: `` git init git checkout -b dev `` or to move an existing repository to a non-master branch: `` git checkout -b dev git push origin dev:dev git branch -D master git push --delete origin master `` You then will also want to set your default repository branch to `dev` in the settings for that repository: https://help.github.com/en/github/administering-a-repository/setting-the-default-branch = `.github` directory is ignored on GitHub Pages {parent=publish-target-github-pages} It's a GitHub bug/feature: https://github.com/orgs/community/discussions/52252 Maybe we should just ignore the `.github` directory when publishing, otherwise it leads to a broken link on the <`-/dir` directory> listings. = `-/raw` files are downloaded rather than displayed in browser for certain file extensions on GitHub Pages {parent=publish-target-github-pages} {tag=`-/raw` directory} TODO find some upstream discussion. = `--publish-target github-md` {parent=`--publish-target`} This output target publishes Markdown to be viewed directly in a GitHub repository. Sample usage: `` ourbigbook --publish --publish-target github-md `` Some demos: * https://github.com/ourbigbook/docs-md[]: a render of this documentation * https://github.com/cirosantilli/cirosantilli.github.io-md[]: a render of https://github.com/cirosantilli/cirosantilli.github.io[] The generated Markdown should be seen primarily as an output format which attempts to render as closely as possible to the corresponding HTML, not as a directly editable canonical input format, as it contains several constructs which are present only to render nicely. Both nonsplit `.md` documents and the `.md` files produced by <split headers> are included, cross-file links target those Markdown files, and generated `index.md` files are named `README.md` so directory landing pages work naturally on GitHub. GitHub documents a general formatted-text preview threshold of approximately 2 MB: https://docs.github.com/repositories/creating-and-managing-repositories/repository-limits#text-limits However, repository landing pages truncate README files at 512 KiB: https://github.com/orgs/community/discussions/23920 Because this target produces README files, it conservatively applies the lower 512 KiB limit to every generated Markdown file. Where a safe header boundary exists, an oversized nonsplit page is automatically replaced by its split-header version. Its table of contents links included source files to their canonical nonsplit `.md` pages rather than their `-split.md` alternates. If the recursive table of contents would still make a split page too large, only its first child level is included. A single header section that remains too large cannot be split safely, so publication continues with a warning asking for another child header. For testing or a more conservative threshold, `githubMarkdownMaxBytes` may be set under the `github-md` target. Values above the GitHub limit are capped at the built-in limit. The OurBigBook documentation configures this target to publish to https://github.com/ourbigbook/docs-md using <ourbigbook.json/target>. = `--publish-target local` {parent=`--publish-target`} Publish as a local directory that can be zipped and sent to someone else, and then correctly viewed by a browser locally by the receiver. You can then zip it from the Linux command line for example with: `` ourbigbook --publish --publish-target local cd _out/publish/_out zip -r local.zip local `` Maybe we should do the Zip step from the <OurBigBook CLI> as well. There is no Node.js standard library wrapper however apparently: https://stackoverflow.com/questions/15641243/need-to-zip-an-entire-directory-using-node-js = `-S`, `--split-headers` {id=split-headers} {parent=OurBigBook CLI options} {title2=Split headers} = Split header {synonym} Split each header into its own separate HTML output file. This option allows you to keep all headers in a single source file, which is much more convenient than working with a billion separate source files, and let them grow naturally as new information is added, but still be able to get a small output page on the rendered website that contains just the content of the given header. Such split pages: * load faster on the browser * get way better Google PageRank for title hits * allow for full metadata display, e.g.: * <Header metadata section> * https://en.wikipedia.org/wiki/Disqus[Disqus]/https://github.com/giscus/giscus[Giscus] comments For example given an input file called `hello.bigb` and containing: `` = h1 h1 content. A link to another section: \x[h1-1]. == h1 1 h1-1 content. == h1 1 1 h1-1-1 content. == h1 1 2 h1-1-2 content. `` a conversion command: `` ourbigbook --split-headers hello.bigb `` would produce the following output files: * `hello.html`: contains the entire rendered document as usual. Remember that this is called `hello.html` instead of `h1.html` because <the toplevel header>[the toplevel header ID is automatically derived from its filename]. Each header contains a on-hover link to the single-file split version of the header. * `hello-split.html`: contains only the contents directly under `= h1`, but not under any of the subheaders, e.g.: * `h1 content.` appears in this rendered output * `h1-1-1` does not appear in this rendered output The `-split` suffix can be customized with the <\H splitSuffix argument> option. The `-split` suffix is appended in order to differentiate the output path from `hello.html` * `h1-1.html`, `h1-1-1.html`, `h1-1-2.html`: contain only the contents direcly under their headers, analogously to `hello-split.html`, but now we don't need to worry about the input filename and collisiont, and just directly use the ID of each header `--split-headers` is implied by the <--publish option>: the published website will automatically get the split pages. There is no way to turn it off currently. A pull request would be accepted, especially if it offers a <ourbigbook.json> way to do it. Maybe it would be nice to have a more generalized way of setting any CLI option equivalent from the `ourbigbook.json`, and an option `cli` vs `cli-publish` so that `cli-publish` is publish only. Just lazy for now/not enough pressing use case met. = Internal link targets in split headers {parent=split headers} = Cross reference targets in split headers {synonym} By default, all <internal links> point to the non-split version of headers, including those found in split headers. The rationale for this is that it gives readers the most context around the header by simply scrolling. For example, considering the example document at <split headers>, <internal links> such as `\x[h1-1]` would point: * from the non-split `hello.html` to the section in the current non-split file `#h1-1` * from split `hello-split.html` to the same section in non-split file with `hello.html#h1-1` The same applies to <cross file internal links> when there are multiple input files. In order to make the split version be the default for some headers, you can use the <\H splitDefault argument>. This is something that we might consider changing with some option, e.g. keeping the split headers more self contained. But for now, the general feeling is that going to nosplit by default is the best default. = `--stdout` {parent=OurBigBook CLI options} When converting a file, output output to stdout in addition to outputting to a file: `` convert --stdout input.bigb `` The regular output file is also saved. Cannot be used when converting a directory. = `--template` {parent=OurBigBook CLI options} Select a custom https://github.com/harttle/liquidjs[Liquid template] file for the output. If not given, this option defaults to the value of <ourbigbook.json/template>, which if not given defaults to `ourbigbook.liquid.html`. The repository of this documentation for example has a sample `ourbigbook.liquid.html` at: \a[ourbigbook.liquid.html]. If no template is present, the default template at one point was: `` <!doctype html> <html lang=en> <head> <meta charset=utf-8> <title>{{ title }}

= OurBigBook Project {c} = OurBigBook {c} {synonym} \b[https://OurBigBook.com[]]: the best way to publish your scientific knowledge. The \b[OurBigBook Project] is creating the ultimate https://github.com/ourbigbook/ourbigbook[open source] tools to \b[textbooks]/\b[<#personal knowledge bases>]/\b[<#Zettelkasten>]/\b[<#digital gardens>] in the <#learn in public> philosophy. It is our best shot yet at the final \b[real-world <#Encyclopedia Galactica>] by allowing effective \b[<#mindmeld>[mind melding]]/\b[<#collective intelligence>] via the \b[]. \Image[logo.png] {height=150} {title=[Logo of] the } This is the using our

User interface (UI)

Apple (fruit)

Apple (company)

My title