# rss_item

guid: http://crawl.develz.org/wordpress/?p=4916
title: 0.26.1 Bugfix release
author: ebering
url: https://crawl.develz.org/wordpress/0-26-1-bugfix-release
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The 0.26.1 bug fix release is now available <a href="https://crawl.develz.org/download.htm">on the download page</a>. Thanks to all the many contributors who helped with this release, including patches and bug reports!</p>
<p>Updating is recommended for all players. Happy crawling, and report more bugs <a href="https://github.com/crawl/crawl/issues">here</a>!</p>


guid: http://crawl.develz.org/wordpress/?p=4920
title: Trunk Updates 16 February 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-16-february-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>As you read the scroll of trunk update it crumbles to dust&#8230;</em></p>
<p>Hello crawlers and welcome to the first trunk updates post of the 0.27 cycle. Things have commenced with a shebang! and there are a lot of new changes to report on. This update will have some thematic organisation to highlight several big sets of changes: <strong>curse removal</strong>, <strong>stat </strong><strong>changes</strong>, <strong>monster spellbooks</strong>, <strong>adjusted demonspawn facets</strong>, and <strong>two new spells</strong>. There are of course all sorts of tweaks, shifts, changes, and <strong>new spells</strong>. Read on for more!</p>
<h2>Curse Removal</h2>
<p>The system of cursed items is no-more, courtesy of PleasingFungus who finally found the collective remove curse scroll and read it for all of us. Practically, this means the following:</p>
<ul>
<li>All items that generate are uncursed</li>
<li>No mundane negative-plus items generate on the floor (though some special shops still place them)</li>
<li><strong>Scrolls of remove curse</strong>, <strong>rings of attention and teleportation</strong>, and <strong>amulets of innacuracy</strong> are removed.</li>
<li><strong>Ashenzari </strong>is the only source of curses. In current trunk, Ash uses identify scrolls instead; however this is a place-holder and a larger revision to Ashenzari is currently in-progress but not yet in trunk.</li>
<li>The <strong>Scythe of Curses</strong> and <strong>Obsidian axe</strong> now have *Drain instead of *Curse; the <strong>necklace of Bloodlust</strong> no longer has a swap penalty.</li>
</ul>
<h2>Stat changes</h2>
<p>Several tweaks to the stat system (including the impact of weapon stats) have arrived, to simplify the impact of base stats and remove hidden breakpoints and step-downs. Thanks to <strong>hellmonk</strong> and <strong>chequers</strong> for the submissions! Along with these changes are some adjustments to species stats.</p>
<ul>
<li><strong>Dexterity</strong> and <strong>evasion</strong> no longer have hidden step-downs</li>
<li>The percentage damage modifier for <strong>strength </strong>on weapon damage is derandomzied and simplified. 10 strength corresponds to 100% damage, and each point of strength above adds 2.5 percentage points to the modifier (points below 10 remove 2.5 percentage points).</li>
<li><strong>Demigod</strong> base stats are reduced to 9/10/9, and now gain 4 points of the stat of their choice every 3 levels. This will give players almost divine control over their stats.</li>
<li><strong>Gnolls</strong> base stats are reduced to 7/7/7.</li>
<li><strong>Spriggans</strong> lose one point of base dexterity and have their Dodging apt reduced to +3.</li>
</ul>
<h2>Monster Spellbooks</h2>
<p>Monsters with multiple distinct spell books have had their spell sets combined into single sets. For a couple of monsters with multiple iconic sets, new monsters have been created to carry the torch. The full set of changes <a href="https://github.com/crawl/crawl/pull/1689">can be perused in the pull request</a>, here are some of the highlights:</p>
<ul>
<li><strong>Psyche</strong> has a new, more insane, spell set. She can cast Polymorph, Chain of Chaos (hi Xom!), Invisibility, and Cantrip.</li>
<li>The named lords of the hells and pandemonium who summon demons now summon demons specific to their realm: <strong>Cerebov</strong> only summons fiery demons now, <strong>Ereshkigal</strong> only tzitzimimeh, and <strong>Dispater</strong> only hell sentinels.</li>
<li>New monster: <strong>Dread Lich</strong>. Ancient liches now have an earth-conjurations spell book with LCS and Corrosive Bolt, but lich summoned demons are a classic. Enter the <strong>Dread Lich</strong>, with Summon Greater Demon, Paralyse, Invisibility, and Haste; appearing in the same places as ancient liches.</li>
<li><strong>Deep elf mages</strong> are removed, replaced with <strong>deep elf pyromancers</strong> (who have a fire-themed spell book) and <strong>deep elf zephyrmancers</strong> (with an air-themed spell book).</li>
</ul>
<h2>Adjusted Demonspawn Facets</h2>
<p>Thanks to <strong>hellmonk </strong>for continued demonspawn improvement work! This latest round of mutation changes focused on the tier 2 facets, with the following adjustments and a new spellcasting-focused facet (combine with Big Brain &#8220;scales&#8221; for a truly demonic mage experience!).</p>
<ul>
<li>The <strong>mana link</strong> facet is re-ordered. The first rank increases MP Regen, the second rank provides guardian spirit, and the third the mana link.</li>
<li><strong>Foul stench</strong> generates miasma at every rank of the facet (scaling with rank) as well as miasma cloud immunity.</li>
<li>The damage curve of <strong>spines</strong> is adjusted to be a bit better at high level.</li>
<li><strong>Ignite blood</strong> now ignites blood at every rank of the facet, with power scaling on rank.</li>
<li><strong>Demonic guardians</strong> are summoned when the player takes damage, and the tier of the guardian grows with level.</li>
<li>The <strong>icemail</strong> facet shifts from pure AC to AC+SH; at rank 1 it grants a <strong>condensation shield</strong> for 4SH, with ranks 2 and 3 granting 4 AC of icy armour each.</li>
<li>A new facet: <strong>malignant magic</strong>. Demonspawn with this facet attempt to paralyse foes in a radius equal to the rank of the facet whenever they cast a spell; at rank 3 this extends to include wand usage. The power of the paralysis depends on the level of the spell cast.</li>
</ul>
<h2>New Spells</h2>
<p>PleasingFungus has developed two new spells aimed at generating new incentives to hybridise.</p>
<ul>
<li><strong>Animate Armour</strong>. Level 4 Summoning/Earth magic. &#8220;Call forth the spirit of your armour, bringing forth a slow-moving ally that hits harder the heavier your armour is. Wham, pow! (Summon limit is 1, as with CCF.)&#8221;. Animate Armour appears in the book of Earth and the book of Dreams.</li>
<li><strong>Manifold Assault</strong>. Level 4 Translocations. Manifold Assault launches attacks with your melee weapon against 2-4 random enemies in LOS, depending on spellpower. Manifold Assault replaces Passage of Golubria in the book of Spatial Translocations, and appears in the book of the Warp (along with Passage of Golubria).</li>
</ul>
<h2>The rest of the changes</h2>
<p>This has been an active month for trunk, and there is a lot more going on. Read on for the rest!</p>
<ul>
<li><strong>Storm clouds</strong> no longer make noise; making <strong>Qazlal</strong> worship quieter at low piety.</li>
<li><strong>Regeneration</strong> is buffed; one pip of regen grants 1 HP per 10 auts (a normal turn). This is 2.5x as strong as it was before.</li>
<ul>
<li><strong>Trolls</strong> and <strong>Vine Stalkers</strong> are adjusted slightly to compensate.</li>
</ul>
<li>Monsters with <strong>ranged weapons</strong> now have unlimited ammo. No more running that stone giant out of large rocks. (Except nets; infininetting gnolls are reserved for gnollcrawl.)</li>
<li><strong>Phantoms</strong> have less HP, and now <strong>blink the player with them</strong> when they blink from attacking.</li>
<li><strong>Big Kobolds</strong> are re-named <strong>Kobold Brigands</strong> and always spawn with darts (one third of the time curare, yikes!). Normal kobolds can no longer spawn with curare (though they can still pick it up from the ground, if you&#8217;re very unlucky!).</li>
<li>Ranged targeting shows<strong> to-hit chance</strong>, and a monster&#8217;s to-hit chance can be seen in xv.</li>
<li><strong>Haunt</strong> moves from the Necronomicon to the Grand Grimoire</li>
<li><strong>Scarves of shadows</strong> no longer act as a spell de-enhancer</li>
<li><strong>Wands of </strong><strong>disintegrate</strong> are now called <strong>wands of </strong><strong>mindburst</strong>, and do not work on mindless creatures.</li>
<li><strong>Manuals</strong> no longer require an inventory slot; once picked up they boost skill aptitude until used up.</li>
<li><strong>Hats</strong> and <strong>Gloves</strong> can spawn with the stealth ego.</li>
<li><strong>Wands of random effects</strong> are de-spoiled, applying a random buff or debuff to the target (15% chance of a buff), with the list of effects visible in the description.</li>
<li><strong>Ozocubu&#8217;s armour </strong>and <strong>Frozen Ramparts</strong> now break on any player position change.</li>
<li><strong>Ozocubu&#8217;s Refridgeration</strong> is now level 7.</li>
<li><strong>Stone arrow</strong> damage is increased.</li>
<li><strong>Monstrous Menagerie </strong>no longer summons harpies.</li>
<li><strong>Summon Demon</strong> and <strong>Summon Greater Demon</strong> are removed as player spells.</li>
<li><strong>Gadget shops</strong> are removed.</li>
<li><strong>Rings of Stealth </strong>and <strong>Boots of Running</strong> are removed.</li>
</ul>
<p>That&#8217;s all for right now! In addition to the Ashenzari changes, on the horizon are more demonspawn changes, refreshes of Vaults and Snake, and new vaults from returned vaultsmith <strong>nicolae</strong>—plus who knows what other twisted horrors we can dig out of the abyss. Until next time, happy crawling!</p>
<div style="text-align: right;"><em>You feel as if something is helping you. You feel blessed forevermore.</em></div>


guid: http://crawl.develz.org/wordpress/?p=4930
title: Trunk Updates 2 March 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-2-march-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>Ashenzari offers you a vision and a curse of trunk updates&#8230;</em></p>
<p>Hello crawlers and welcome to the first trunk update of March! It&#8217;s been busy here in Coc:7 dreaming up new and varied ways of torturing players. The first thing of note in this round of trunk updates is an <strong>Ashenzari overhaul:</strong></p>
<ul>
<li>Ashenzari periodically offers followers <strong>curses</strong> as they explore.</li>
<li><strong>Curses</strong> are applied to equipped items and have the following effects:</li>
<ul>
<li>Bind the item to you, so it cannot be removed,</li>
<li>Increase your piety with Ashenzari,</li>
<li>and grant a boost to two groups of skills (described when Ashenzari offers the curse).</li>
</ul>
<li>Curses can be removed at any time by <strong>shattering the chains</strong>. This <em>destroys</em> the item, freeing you from the curse.</li>
<li>In exchange for binding yourself with curses, Ashenzari grants the following benefits:</li>
<ul>
<li><strong>Skill boosts</strong> granted by your curses grow with piety.</li>
<li>At 0* Ashenzari reveals a <strong>wealth of knowledge</strong>: all items (including consumables) are identified, exploration traps are prevented, and portals are revealed.</li>
<li>Passive <strong>monster and item </strong>detection and <strong>passive mapping</strong>, with detection range scaling with piety.</li>
<li>At 2* <strong>See Invisible</strong></li>
<li>At 3* <strong>Clarity</strong></li>
<li>Starting at 4* <strong>passive scrying</strong> with radius 2, scaling up to radius 4 at 6*.</li>
</ul>
</ul>
<p>Other changes since we last wrote:</p>
<ul>
<li>Plain <strong>Liches</strong> have a new spell set: Bolt of Cold, Orb of Destruction, Slow, Haste and Invisibility; and <strong>Ancient Liches</strong> trade Slow for Petrify.</li>
<li>If he spawns on the orb run, <strong>Boris </strong>will do so with permanent haste and a permanent aura of brilliance, spurred on by his lust for the ORB.</li>
<li><strong>Airstrike</strong> no longer does bonus damage to flying things.</li>
<li>The <strong>obsidian axe</strong> cannot be unwielded while mesmerised; watch out!</li>
<li>Once you complete a <strong>Ziggurat</strong>, the Zot clock no longer applies.</li>
<li>The <strong>spectral brand</strong> has been simplified: The weapon pops out when you attack, immediately follows up, and then sticks around until the end of your next turn. If you hit again, everything repeats. Otherwise, it vanishes.</li>
<li><strong>Unidentified equipment</strong> is now identified when you touch the cell that it is on, instead of on wear.</li>
<li>The <strong>Depths</strong> monster list has been trimmed of so-called &#8220;trash mobs&#8221; for a deadlier experience.</li>
<li><strong>*Slow</strong> as an artefact property now stacks, multiple sources will increase the chances of getting slowed when hit.</li>
</ul>
<p>That&#8217;s all for now! On the ever changing horizon, changes to Snake and the Vaults seem in focus, but it is always hard to see through the impenetrable haze at the border between Cocytus and Gehenna, so who can say, really? Until next time, happy crawling!</p>
<div style="text-align: right;"><em>You shatter the chains binding the trunk update!</em></div>


guid: http://crawl.develz.org/wordpress/?p=4937
title: Trunk Updates 23 March 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-23-march-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>A trunk update comes into view. Wyrmbane glows green as a worthy foe approaches.</em></p>
<p>Hello crawlers, it&#8217;s been a busy few weeks in the cottage industries of the Iron City. This trunk update has several big things, but before we get to them I want to highlight some non-code community submissions since we last wrote:</p>
<ul>
<li>New <strong>vaults</strong> by <em>beargit, <em>n</em><em>icolae, </em></em>and <em>Skrybe</em>, appearing in various places in the dungeon. If you&#8217;re interested in making vaults (prefabricated bits of the map), read an introduction to editing vaults <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/levels/introduction.txt">here</a> and consider setting up a <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/git/quickstart.txt">development environment to test your vaults</a>.</li>
<li>New <strong>tiles</strong> by <em>Sastrei</em>. If you&#8217;re interested in making tiles, read our tiles guideline <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/tiles_creation.txt">here</a>.</li>
</ul>
<p>Both kinds of contribution can be submitted on our <a href="https://github.com/crawl/crawl">GitHub repository</a>, either as an issue or a pull request, if you don&#8217;t have a mantis account. If the documents seem a little old, they&#8217;ll be getting updates in the coming month.</p>
<p>Diving into some of the other changes that have happened since we last wrote. <strong>Several branches</strong> have had changes.</p>
<div>
<ul>
<li><strong>Slime</strong> gains two new monsters:</li>
<ul>
<li><strong>rockslimes</strong>, a high AC flame resistant slime that tramples and hits like a rock.</li>
<li><strong>quicksilver oozes,</strong> an antimagic and engulfing magic-immune ooze.</li>
<li><strong>Eyes of draining</strong> and <strong>death oozes</strong> are removed (both from Slime and the game).</li>
</ul>
<li><strong>Vaults</strong> monster sets have been heavily revised:</li>
<ul>
<li>so-called &#8220;trash mobs&#8221; no longer spawn, for a deadlier experience.</li>
<li><strong>orc warlords</strong> in Vaults come with bands of loyal knights, and nothing weaker.</li>
<li><strong>wizards</strong> in Vaults come with bands of ugly things, their disgusting pets.</li>
<li>New <strong>ironbound </strong><strong>frosthearts</strong> are human spellcasters see invisible and draw creeping frost from the walls.</li>
<li>New <strong>ironbound thunderhulks</strong> are ogres with a magical lightning rod, which can fire smite-targeted lightning, but cannot hit nearby foes.</li>
<li>New <strong>ironbound </strong><strong>beastmasters</strong> lead beastly war bands of dire elephants or lindwurms through the vaults, speeding them along with spiked goads.</li>
<li><strong>peacekeepers</strong> have arrived to keep the peace in the vaults, coming with bands of constructs.</li>
</ul>
<li><strong>Snake</strong> monsters have been refreshed (in addition to a change to poison mechanics):</li>
<ul>
<li><strong>naga</strong> and <strong>naga warriors</strong> get a buff to base damage, and the latter have learned a battlecry.</li>
<li><strong>naga ritualists</strong> no longer cast force lance and have become priests.</li>
<li><strong>nagaraja</strong> no longer cast force lance or teleport other, and now cast dimension anchor.</li>
<li><strong>naga magi</strong> now cast venom bolt, haste, and enchant their allies to turn their poison into curare.</li>
<li>New <strong>salamander tyrants</strong> call forth volcanic eruptions and glare with a weakening gaze.</li>
</ul>
<li>The <strong>early dungeon</strong> has some new features:</li>
<ul>
<li><strong>Shops</strong> can generate on D:1-3.</li>
<li><strong>Rare scrolls, books, manuals, and staves</strong> can generate on D:1-3 at normal rates.</li>
<li><strong>Quokkas</strong> can spawn on D:1.</li>
<li><strong>Ball pythons</strong> are adjusted.</li>
<li><strong>Hobgoblins</strong> no longer spawn with a weapon but have higher base damage.</li>
<li><strong>Oozes</strong> have less hp and melee damage, but can cast freeze.</li>
<li><strong>Leopard geckos</strong> are removed.</li>
</ul>
</ul>
<p>Next, a <strong>new species</strong> has arrived in trunk: <strong>Djinn. </strong> Djinni are floating spirits formed from smokeless fire, they have no MP, instead casting spells using their HP. They also cannot learn spells from books; they instead get spells randomly and naturally, starting on XL 3 and continuing on every even level. The former mechanic is inspired by a design from kilobyte, who created another &#8216;Djinni&#8217; species in 0.13. The latter comes from pubby, who created an &#8216;Oni&#8217; race using it in various DCSS forks. We are indebted to their ideas, though this implementation is somewhat different from both.</p>
</div>
<p>Of course, trunk development is not all headline items. There&#8217;s been a lot of behind the scenes technical work, bug fixing, and an assortment of other changes. Read on to see what else has changed.</p>
<ul>
<li><strong>Wyrmbane</strong> now glows when a dragon that can upgrade it comes into view.</li>
<li><strong>Dexterity and EV</strong> formulas are tweaked again.</li>
<li>Assorted <strong>autoexplore</strong> edge-case improvements by Aliscans.</li>
<li><strong>Swamp</strong> is now loud.</li>
<li><strong>Cheibriados</strong> no longer hates quickblades.</li>
<li><strong>Elixir</strong> gives both heath and mp at all power levels.</li>
<li><strong>Zin sanctuary</strong> no longer applies holy word.</li>
<li>Empty space no longer reduces the variance of <strong>Airstrike</strong>&#8216;s damage.</li>
<li><strong>Kobold Fighters</strong> start with bucklers now.</li>
<li>Monster <strong>weapon delay</strong> is standardized at the monster&#8217;s speed, certain monster<br />
base damages are adjusted to compensate. (hellmonk)</li>
<li><strong>Poisonous Vapours</strong> can no longer target empty space.</li>
<li><strong>Palentonga</strong> HP apt is reduced to +0.</li>
<li>Player <strong>poison</strong> now ticks much faster.</li>
<li><strong>Yredelemnul</strong> accepts nonliving worshippers again. Return the black torches to your Gargoyles and statuary!</li>
<li><strong>Halflings </strong>are removed.</li>
<li><strong>Shatter</strong> deals bonus damage to<strong> toenail golems</strong> (youch).</li>
</ul>
<p>That&#8217;s all for now. The future keeps approaching even though we can never see it clearly, but by desperate Aperomancy it seems that the branch freshening trend might reach to Spider or the Hells. More vaults from vault smiths new and old are soon to appear. Perhaps hellmonk&#8217;s promised Tier 3 demonspawn changes will materialize? Or perhaps none of these things will happen and this space will show something completely different next time! Until then, happy crawling!</p>
<p style="text-align: right;"><em>The trunk updates whisper &#8216;farewell&#8217; as they head to the true west.</em></p>


guid: http://crawl.develz.org/wordpress/?p=4944
title: Trunk Updates 2 April 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-2-april-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>You pray at the faded altar. The Trunk Updates accept your prayer!</em></p>
<p>Hello crawlers, trunk is currently as hot as Geh:7 on a hot day with lots of commits coming fast. Since we last wrote over 150 changes have landed in trunk, a mix of tweaks, shifts, new features, community contributions, and bugfixes. As with last time, I want to again highlight our non-code contributors:</p>
<ul>
<li>New <strong>vaults</strong> by <em>beargit, <em>n</em><em>icolae, </em></em>and <em>ShaperMC</em>, appearing in various places in the dungeon. If you&#8217;re interested in making vaults (prefabricated bits of the map), read an introduction to editing vaults <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/levels/introduction.txt">here</a> and consider setting up a <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/git/quickstart.txt">development environment to test your vaults</a>.</li>
<li>New <strong>tiles</strong> by <em>Sastreii</em>. If you&#8217;re interested in making tiles, read our tiles guideline <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/develop/tiles_creation.txt">here</a>.</li>
</ul>
<p>Both kinds of contribution can be submitted on our <a href="https://github.com/crawl/crawl">GitHub repository</a>, either as an issue or a pull request, if you don&#8217;t have a mantis account. If the documents seem a little old, they&#8217;ll be getting updates in the coming month.</p>
<p>Work on spicing up the <strong>early dungeon experience</strong> has continued with significant changes to the <strong>overflow altar system</strong>:</p>
<ul>
<li>Each floor from D:1-3 has a 50% chance of placing a <strong>faded altar</strong>. <em>We need more faded altar vaults! If you&#8217;re interested in making vaults this is a good place to start (as are arrival vaults).</em></li>
<li>There is a 20% chance of an identified `altar to a <strong>random Temple god</strong> on D:2.</li>
<li>Except for the previous two chances, no other altars appear on D:1 or D:2.</li>
<li>Every &#8220;Temple god&#8221; receives <strong>exactly one guaranteed altar</strong>, appearing either in the<strong> Ecumenical Temple</strong> or in an <strong>overflow temple</strong> <strong>between D:3 and D:10.</strong> <em>We also need more overflow temple vaults!</em></li>
</ul>
<p>In addition to the changes to altar placement, further adjustments continue:</p>
<ul>
<li><strong>Oozes</strong> are renamed <strong>endoplasms</strong></li>
<li><strong>Dungeon</strong> spawn tables are adjusted to produce more depth appropriate monsters<br />
and less chaff.</li>
<li><strong>Ogres</strong> and <strong>Two headed ogres</strong> again attack slowly and hit harder.</li>
<li><strong>Scorpions</strong> are now fast.</li>
<li><strong>Gnolls</strong> on D:1 no longer get <strong>halberds</strong>.</li>
<li><strong>Worker ants</strong>, <strong>soldier ants</strong>, and<strong> queen ants</strong> are removed.</li>
</ul>
<p>The new <strong>Djinni</strong> species continues to be tweaked in response to play testing:</p>
<ul>
<li><strong>Weapon aptitudes</strong> have been lowered by one.</li>
<li>The <strong>scholarly gods</strong> now all reject Djinn worship (that&#8217;s <strong>Sif Muna</strong>, <strong>Vehumet</strong>, and <strong>Kikubaaqudgha</strong>)</li>
<li>Spells are guaranteed on level-up: if a starting book spell is rolled the spell gets re-rolled until an available spell is found.</li>
</ul>
<p>Some changes to the <strong>mutation pool</strong>:</p>
<ul>
<li>All <strong>scales</strong> except <strong>Iridescent</strong> scales are now Demonspawn only.</li>
<li><strong>Iridescent scales</strong>, <strong>bone plates</strong>, <strong>thin skeletal structure</strong>, and <strong>sturdy frame </strong>are no longer incompatible.</li>
<li><strong>Blurry vision</strong> is removed, replaced by allowing Ru&#8217;s <strong>-Scroll</strong> while threatened mutation in the random mutation pool.</li>
<li><strong>Shoutitis</strong> and <strong>No potion heal</strong> are reduced to 2 levels, with the maximum level effect at the top level, and half the effect at the first level.</li>
<li>A new <strong>Jiyva mutation</strong> gives player melee an <strong>engulf</strong> effect, engulfing foes in ooze (gross).</li>
<li>The <strong>blink mutation</strong> is removed.</li>
</ul>
<p>And of course, a grab bag of other changes:</p>
<ul>
<li><strong>Manifold Assault</strong> is raised to level 5.</li>
<li><strong>Depths vaults</strong> have their monster lists updated to match the spawn table.</li>
<li>The <strong>Vestibule of Hell</strong> has its monster list updated to remove chaff monsters.</li>
<li><strong>Flying equipment</strong>, <strong>species</strong>, and <strong>forms</strong> cause you to fly at all times.</li>
<li><strong>Sif Muna&#8217;s</strong> book gifting rate has been reduced, to make it more feasible to reach 6* piety.</li>
<li><strong>Blade Hands</strong> is now level 6 and appears only in the book of Transfigurations</li>
<li><strong>Hydra Form</strong> is now level 5 and appears in the book of Changes and the Fen Folio</li>
<li><strong>Heavenly Storm</strong> grants bonus EV and is a bit quieter.</li>
<li><strong>Scrolls of random effects</strong> are removed.</li>
</ul>
<p>That&#8217;s all for now. On the horizon: new demonspawn tier 3 facets have come into focus. The winds of freshening continue to blow through branches but just where they&#8217;ll go remains mysterious. Until next time, happy crawling!</p>
<p style="text-align: right;"><em>You feel </em><em>threatened and lose the ability to read!</em></p>


guid: http://crawl.develz.org/wordpress/?p=4955
title: Trunk Updates 19 April 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-19-april-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The trunk updates feel delightfully devilish&#8230;</em></p>
<p>Hello crawlers, and welcome to another installment of the 0.27 trunk updates!</p>
<p>A big thanks to <em>n</em><em>icolae </em>for another big batch of vaults, this time for <strong>Spider</strong>; to <em>Dilly</em>, <em>ShaperMC</em>, and <em>hellmonk </em>for <strong>new ecumenical altar vaults</strong>; and to Sastreii for a continued stream of new tiles! Additionally a big thanks to <em>hellmonk </em>for the overhaul to <strong>Demonspawn Tier 3 facets</strong>:</p>
<ul>
<li>The <strong>Black mark</strong> facet gives the following benefits. At level 1 hex enhancer, level 2 black mark, and at level 3 a radius 1 silence donut around the player (the player isn&#8217;t silenced).</li>
<li>The <strong>Damnation </strong>facet is adjusted to feel more like becoming a Brimstone fiend. At level 1 the facet grants Will+ and passive retaliation to hex attempts, at level 2 gives 50% torment resistance, and at level 3 hurl damnation (which has some tweaks to its damage scaling).</li>
<li>A new <strong>Chaos facet</strong> which gives at level 1 a chance to corrode enemies when you damage them, at level 2 a chance to malmutate enemies when you damage them, and at level 3 an active ability to forcibly blink away and apply negative statuses to enemies in line of sight.</li>
<li>The <strong>Robust</strong> and <strong>Augmentation</strong> facets are unchanged; <strong>Passive freeze</strong> is removed, and the passive freeze mutation moves into the general pool.</li>
</ul>
<p>Read on for the rest of the trunk changes:</p>
<ul>
<li><strong>New Unrand:</strong> the seven-league boots, which grant a full-LOS rampaging effect</li>
<li><strong>Yredelemnul</strong> again permits statue form</li>
<li><strong>Nemelex&#8217; Foxfire card</strong> is renamed &#8220;the Swarm&#8221; and summons a swarm of bees with size varying on power.</li>
<li>More <strong>salamanders</strong> have been placed in Snake rune vaults</li>
<li><strong>Draconian Knights</strong> and <strong>Fannar</strong> no longer cast Ozocubu&#8217;s Armour and have higher base AC.</li>
<li><strong>Ice elementalists</strong> now start with Metabolic Englaciation instead of Summon Ice Beast</li>
<li><strong>Hailstorm</strong> now affects icy creatures</li>
<li><strong>Wands of random effects</strong> are removed, replaced with the XP-charged <strong>pieces from Xom&#8217;s chessboard</strong>, which apply a random effect to a random target in LOS.</li>
<li>The diversity of early dungeon books is increased.</li>
<li><strong>Static Discharge</strong> is lowered to level 2, and <strong>Swiftness</strong> increased to level 3.</li>
<li><strong>Mnoleg</strong> loses Random Bolt, to better focus on summoning eyeballs.</li>
<li>The <strong>demonic rune</strong> is announced when a vault that is guaranteed to place it is generated.</li>
<li>The player and their <strong>Hepliaklqana ancestor</strong> can now fire through one another.</li>
<li><strong>Major Destruction</strong> is buffed and has the list of zap adjusted. <strong>Makhleb&#8217;s</strong> passive heal on kill is reduced in strength.</li>
<li><strong>Vehumet</strong> now supports Poisonous Vapours.</li>
</ul>
<p>That&#8217;s all for now! As always, the future is hazy; there are some big changes proposed by community contributors that are under discussion, as well as some big UI projects under way. One of those projects, by long time contributor <em>nlavsky</em> is nearing completion: it will bring an inventory panel to WebTiles! Until next time, happy crawling!</p>
<p style="text-align: right;"><em>You make a move on the chessboard. The trunk updates cackle and vanish!</em></p>
<p>&nbsp;</p>


guid: http://crawl.develz.org/wordpress/?p=4970
title: IRC presence moved to libera.chat
author: advil
url: https://crawl.develz.org/wordpress/irc-presence-moved-to-libera-chat
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Because of the change in ownership in Freenode, over the last few weeks we have migrated the DCSS IRC channels and infrastructure, including the Sequell/Cheibriados infobots, to <a href="https://web.libera.chat/#crawl">#crawl</a> and #crawl-dev on <a href="https://libera.chat/">libera.chat</a>. (Note the single &#8220;#&#8221;!)</p>
<p>Also: we had planned to maintain a presence on old freenode in order to point visitors to the correct new place. However, as of 6/15 the new owners have apparently <a href="https://www.devever.net/~hl/freenode_suicide">wiped</a> the network&#8217;s nickserv and chanserv databases entirely, so we no longer have control over our old channels (or identities) and are not using them at all.</p>


guid: http://crawl.develz.org/wordpress/?p=4974
title: Trunk Updates 25 June 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-25-june-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>Welcome to Tudoco Ucre&#8217;s Tempting Trunk Update!</em></p>
<p>Hello crawlers, it has been a while since a post has appeared in this space. While your author has been inactive the rest of the team and the community have been working away in the depths of Gehenna to sift the most lethal ideas from the hellish magma.</p>
<p>First of all, a warm welcome to the two newest players for the Devil&#8217;s team: <em>hellmonk–</em>a long time community contributor, fork developer, and notorious memelord. <em>nicolae–</em>another long time community vaultsmith, with a special talent for shop vaults, whimsey, and good quotations. We&#8217;re happy to have both of them aboard the team.</p>
<p>Speaking of community contributors, since I last wrote from among the 285 commits to trunk we&#8217;ve seen</p>
<ul>
<li>A new splash screen by <em>ArbitUHHH</em></li>
<li>New tiles from <em>Sastreii, </em>including for <strong>demonic trees</strong> which inspired a new dungeon feature</li>
<li>Tweaks and fixes from <em>Goratrix, dilly, nlavsky, and DreamDust</em></li>
<li>Specific mechanical input from <em>engine</em> and <em>pubby</em></li>
</ul>
<div>On to the changes!</div>
<h2>New and Improved Vaults and Environment</h2>
<ul>
<li>Everyone&#8217;s favorite lair ending, <strong>the Jungle Book</strong>, has been overhauled to no longer be a Lua-trigger spoiler. There are still plenty of appearing monsters, brought back from the dead by necromancers that now appear.</li>
<li><em>hellmonk</em> has been on fire, adding three new <strong>Volcanoes</strong>: a smoke and steam themed map, a tribute to the Zot hot tub (beware the orb of fire), and an elven volcano full of pryromancers.</li>
<li>To cool off, he then added a slime themed new <strong>Ice Cave</strong>.</li>
<li>Both <em>hellmonk</em> and <em>nicolae </em>have been at work improving <strong>Snake </strong>vaults; the first with a new <strong>rune vault</strong>, and the second with a wide variety of thematic vaults.</li>
<li>Many old <strong>guarded unrand</strong> vaults have been tweaked and adjusted, as well as a slew of new ones.</li>
<li>While not exactly a vault: <strong>demonic trees</strong> now appear in the Abyss, Pan, and themed vaults. They can be awakened like normal trees but when burned produce a cloud of chaos instead of a forest fire.</li>
<li>Early <strong>out of depth monsters</strong> have a shallower maximum depth.</li>
</ul>
<h2>Gods</h2>
<ul>
<li><strong>Hepliaklqana</strong> waits until you have shown enough devotion to earn 1* of piety to bring your ancestor out of the mists of memory (and kindly also waits to impose the HP penalty).</li>
<li><strong>Sacrifice Experience</strong> costs more XP now: the experience cost of skills is calculated with your old XL value, which will have a much longer effect. The sacrifice can now be repeated up to three times.</li>
<li><strong>Sacrifice words</strong> and <strong>sacrifice drink</strong> grant bonus piety when both are taken (Mummies get the bonus just for sacrificing words.)</li>
<li>Other <strong>sacrifice piety values</strong> have been tweaked and rebalanced in various directions.</li>
<li><strong>Okawaru</strong> no longer cares if you slaughter your allies.</li>
<li><strong>Kikubaaqudgha </strong>now gifts spells directly into your library; the capstone grants all of the spells of the Necronomicon and also Haunt once more.</li>
</ul>
<h2>Character</h2>
<ul>
<li><strong>Djinni</strong> aptitudes are adjusted (XP is increased from 0 to +1 and spellcasting reduced from -2 to -3).</li>
<li><strong>Unbreathing </strong>is no longer a mechanic distinct from rPois or holiness type (this applies also to monsters).</li>
<li><strong>Palentonga</strong> can now silence their rolls.</li>
<li><strong>Wisp form</strong> no longer has blinkitis.</li>
<li>The <strong>guaranteed damage reduction mechanic</strong> is changed. Instead of being a secret percentage associated with particular armour items, it is an internal calculation derived from AC from any source. For the curious <a href="https://github.com/crawl/crawl/commit/ff078d554e3f">you can read the details</a>, but to quote the LearnDB &#8220;If you&#8217;re making equipment decisions based on this, please reconsider.&#8221;</li>
</ul>
<h2>Spells</h2>
<ul>
<li>The <strong>amulet of the Air </strong>loses EV+5 and becomes an Air magic enhancer.</li>
<li>Monster <strong>Bind Souls</strong> now always creates a simulacrum when the targeted monster dies.</li>
<li><strong>Air Elementals</strong> lose their twisters, and gain Stunning Burst which inflicts 1 turn of paralysis and is resistible with rElec, stasis, or by dodging the attack (but not Will).</li>
<li><strong>Ice form</strong>&#8216;s AC scaling with power is buffed.</li>
<li><strong>New spell: Storm Form </strong>Level 7 Air/Transmutations. Transforms the caster into a storm, melding all non-jewellery, and granting power-scaling electric cleaving melee, power-scaling defenses, and an activated blinkbolt ability.</li>
<li><strong>Wereblood</strong> grants a small amount of healing on each kill.</li>
<li><strong>Blade Hands</strong> is again level 5.</li>
<li><strong>Freezing Cloud</strong> is now level 5.</li>
<li><strong>Tornado</strong> is re-named and re-flavoured to <strong>Polar Vortex</strong> Level 9 Ice; the damage flavour is changed to icicle damage and increased.</li>
<li><strong>Chain Lightning</strong> is increased to Level 9 Conjurations/Air, and has its mechanics simplified to be easier to reason about (but still a bit more complicated than a bullet point).</li>
<li><strong>Poisonous Vapours</strong> no longer creates a cloud as part of its mechanical function.</li>
<li><strong>Absolute Zero </strong>and <strong>Hydra Form</strong> are removed.</li>
</ul>
<div style="text-align: right;"><em>The trunk updates go straight to hell!</em></div>


guid: http://crawl.develz.org/wordpress/?p=4980
title: Trunk Updates 1 July 2021 and Tournament Date Set
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-1-july-2021-and-tournament-date-set
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>You open the Incunabulum of Trunk Updates and begin reading&#8230;</em></p>
<p>Hello crawlers! Coming hot on the heels of our previous trunk update we&#8217;re back with more of the latest. Before we dive into changes, some exciting scheduling matters:</p>
<p style="text-align: center;"><strong>The 0.27 </strong><strong>tournament is tentatively scheduled for July 30th-Aug 15th</strong></p>
<p style="text-align: left;">As usual, watch this space for links to the tournament webpage, more information about scoring, etc.</p>
<p style="text-align: left;">While it has only been a week since our last little chat, almost 100 commits have landed in trunk. One significant area of change is to spellbooks, feedback on the feel of this change is most welcome and we look forward to reading your remarks on various crawl forums. With that said, on to the changes!</p>
<ul>
<li>Thanks to community contributor Aliscans, <strong>wizmode monster creation</strong> now has in-game help, for all your monster experimentation needs.</li>
<li><strong>Archmagi</strong> can now appear on randart robes.</li>
<li>Several <strong>unrands</strong> have been adjusted:</li>
<ul>
<li><strong>Damnation</strong> only consumes bolts and explodes on hit.</li>
<li>the staff of <strong>Wucad-Mu</strong> has been buffed, refunding MP more often and including the refund even when it backfires.</li>
<li>the <strong>Wrath of Trog</strong> is angrier: it now extends your berserk duration on kill (sometimes) and grants Rampage. Raarhrhrhghghghghgh!</li>
</ul>
<li>The <strong>post-berserk penalty</strong> no longer includes paralysis.</li>
<li>Using <strong>Trog&#8217;s Berserk</strong> now has a piety-adjusted chance to fail.</li>
<li>Salamander Tyrants, Rockslimes, and Quicksilver oozes have been added to <strong>Ziggurat monster sets.</strong></li>
<li>Several further enhancements to the <strong>quiver</strong>:</li>
<ul>
<li>Potions and scrolls can now be quivered for use.</li>
<li>Autofight correctly supports &#8220;waiting spells&#8221; like searing ray.</li>
<li>New options are available to customize the quiver order</li>
</ul>
<li>A <strong>new spell:</strong> Maxwell&#8217;s Capacitive Coupling (Level 8 Air). When cast, the player begins charging up. If the player waits (with &#8216;.&#8217;) and is not moved then after a bit of time (random and depending on spell power) the accumulated charge is discharged, vaporizing the nearest foe (nearest at the end of the spell, not when it is cast!). This might seem familiar, but I&#8217;ve got <em>absolutely zero</em> idea where you might have seen the mechanics before.</li>
<li><strong>Wereblood</strong> now only grants its healing for kills of adjacent monsters.</li>
<li>Felids no longer get a stealth penalty from <strong>Blade Hands.</strong></li>
<li>Everyone else can improve their stealth with a <strong>new good mutation</strong> that dampens sounds.</li>
<li><strong>Spell book changes:</strong></li>
<ul>
<li>Spellbooks are always identified</li>
<li><strong>Randart spellbooks</strong> have fewer spells</li>
<li><strong>Fixed spellbooks</strong> are split into smaller books of 2-4 spells each, and <em>30 new books</em> are added to pick up the contents.</li>
<li>Spellbooks generate <strong>more than twice as often</strong> to keep the number of spells generated about the same. Book rarity is now more closely tied to the spell level.</li>
<li><strong>Book pricing </strong>in shops is reworked</li>
<li><strong>Backgrounds</strong> that previously started with a spell book are unchanged, their starting spells are added directly to the spell library at game start.</li>
</ul>
</ul>
<p>That&#8217;s all for now. In the run up to the release and tournament, &#8220;feature freeze&#8221; is scheduled roughly around July 16 or 23. Before then expect to see many more exciting new vaults reach trunk, both from our new devteam members and from several community contributors; along with balance tweaks, bug fixes, and user interface improvements (including, hopefully, an item panel in the WebTiles UI). Until next time, happy crawling!</p>
<p style="text-align: right;"><em>Weird images run through your mind. The Trunk Updates vapourize in an electric haze!</em></p>


guid: http://crawl.develz.org/wordpress/?p=4987
title: July 24 Trunk Update Post and 0.27 Tournament Page
author: gammafunk
url: https://crawl.develz.org/wordpress/july-24-trunk-update-post-and-0-27-tournament-page
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>A +6 arbalest &#8220;Damnation&#8221; {damnation} comes into view. The +6 arbalest &#8220;Damnation&#8221; {damnation} shoots a damnation bolt at the trunk update.</em></p>
<p>Hello crawlers and welcome to the last trunk update of the 0.27 cycle. We&#8217;ve been under <strong>feature freeze</strong> since last Friday, so the remaining time before the <strong>0.27 Release and Tournament</strong> will be devoted to bug fixes, balance tweaks, and polish. If you&#8217;ve been holding out and playing 0.26 due to concerns about stability, know that all major features are fixed in trunk, and only reasonable balance changes and bug fixes will be a thing from here until release. We&#8217;ve made the 0.27 beta branch, so servers will begin installing 0.27 soon. So please play-test trunk or the 0.27 branch when that&#8217;s available on your favorite server to help us find bugs! The 0.27 release will be either Thursday, July 29th, or Friday, July 30th, depending on timing. Look out for an official release post with download links to come.</p>
<p>The <strong>0.27 tournament</strong> will run from <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210730T20">Friday July 30th, 8pm UTC</a> to <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210815T20">Sunday August 15th, 8pm UTC</a>. Please bookmark the <strong><a href="https://crawl.develz.org/tournament/0.27/">official 0.27 tournament page</a> </strong>and use this page to track your progress throughout the tournament. We&#8217;ll continue to update this page with any rules changes. In coming days you may see summaries of test data in the &#8220;Overview&#8221; page taken from recent 0.27 beta games. Don&#8217;t worry, we&#8217;ll reset everything before the tournament starts.</p>
<p>Now, onto the Trunk Update! Note that this is not the full set of 0.27 changes, just the ones since our last trunk update. For the full and official set of 0.27 changes, check out the <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/changelog.txt">changelog</a>. Here are the changes since last time:</p>
<ul>
<li>The<strong> starting spell sets</strong> of some Mage and Warrior-Mage backgrounds are reduced by one spell. This together with the increase in book drops will give these classes more variety in how they proceed through middle portions of the game. We didn&#8217;t get to adjusting all classes this version, since for some, any spell removal would leave the class in a difficult spot. Further changes, including potentially new spells, will happen during 0.28 development, so stay tuned! The changes for 0.27:</li>
<ul>
<li>Air elementalists no longer start with Lightning Bolt.</li>
<li>Arcane Marksmen no longer start with Leda&#8217;s Liquefaction.</li>
<li>Earth Elementalists no longer start with Lee&#8217;s Rapid Deconstruction.</li>
<li>Summoners and Ice Elementalists no longer start with Summon Ice beast.</li>
<li>Venom Mages no longer start with Ignite Poison.</li>
</ul>
<li><strong>New branch end maps and adjustments:</strong></li>
<ul>
<li>New &#8220;Abyssal Woods&#8221; Lair ending featuring demonic trees and abyss-themed monsters. Beware the thrashing horrors!</li>
<li>New &#8220;Lost City&#8221; Shoals ending featuring a ruined temple theme with minotaurs and sphinxes. Beware the satyrs too!</li>
<li>New &#8220;Storm Temple&#8221; Shoals ending featuring electricity-themed monsters and many possible layouts. Beware the storm dragon! Or the titan!</li>
<li>The &#8220;Ancient Temple&#8221;,  &#8220;Frog Pond&#8221;, and &#8220;Undead Jungle Book&#8221; Lair endings have been adjusted to lower monster counts significantly. And they say we never give players anything&#8230;</li>
<li>Most Snake end maps have their monster counts significantly reduced. It&#8217;s tough work slogging through all those nagas, we know.</li>
</ul>
<li>All<strong> Ice Cave</strong> maps have been rebalanced to have more varied layouts, monsters, and loot. Gone are mysterious dead reapers! The changes are substantial, as detailed in <a href="https://github.com/crawl/crawl/commit/cc58b11c1d06">this commit</a>. You should find the cloud generator placement less arbitrary and less frequent, more possible shortcuts in many maps, and more varied monster sets at both easy and hard difficulty ice caves. Stay frosty, friends!</li>
<li>The <strong>Robe of Folly</strong> is redesigned. It now gives you a permanent brilliance effect (spell enhancer and half spell MP costs), but sets your Will to 0. Additionally it grants +8 Int (from +4) and +4 enchant (from +3). It&#8217;d be folly indeed to not wear this robe while entering one of my lich-filled vaults!</li>
<li><strong>Dancing ranged weapons</strong> are now a thing that can kill you. Yes, <em>of course</em> they have infinite ammo, I&#8217;m glad you asked! But don&#8217;t worry because&#8230;</li>
<li>&#8230;<strong>Tukima&#8217;s Dance</strong> now works on monsters wielding ranged weapons! It can make charmed dancing ranged weapons to fight by your side.</li>
<li>The <strong>Elemental Staff</strong> is featured in one new and one reworked vault. Both vaults also feature a greatly improved <em>master elementalist </em>monster that always wields the staff, appearing when these vaults place in Elf or Depths. Watch out, it knows Ozo&#8217;s Refridge, Crystal Spear, Chain Lightning, and Firestorm!</li>
<ul>
<li>The new vault is the official &#8220;guarded unrand&#8221; vault, by first-time contributor dilly. Placing in Dungeon, Elf, and Depths, it features an elemental mage elf boss (for Dungeon) or the master elementalist (for Elf and Depths) and various elemental baddies.</li>
<li>Mu&#8217;s &#8220;elemental laboratory&#8221; vault, the one that initially introduced the master elementalist, has been given a complete overhaul. It has more challenging and interesting elemental monster sets and features much more interesting terrain.</li>
</ul>
<li><strong>Monster breath</strong> timeouts now reduce after their turn. So if you see that a monster is &#8220;catching its breath&#8221;, you know it won&#8217;t use a breath attack before your next turn unless it gets a double turn. That&#8217;s really nice!</li>
<li>Many <strong>new vaults</strong> for Elf, including entries, decorative vaults, monster vaults with weird things like defective dancing weapons and cool things like double deep elf blademasters with double double swords, and new main vaults and entries for the Elf:2 Hall of Blades! Also new are a batch of ecumenical altar vaults. New even though the altars inside are very old.</li>
<li><strong>Eleionomae </strong>no longer land on and destroy trees when they travel next to you, instead picking a safe and comfy spot next to a tree. No more tree digging!</li>
<li><strong>Ossuaries</strong> no longer have spear and dart traps. The days of step-and-rest-then-repeat for clearing some areas of ossuaries are over! Don&#8217;t worry, alarm traps and net traps still make an appearance. And there&#8217;s talk of introducing some new types of traps or trap-like monsters in 0.28&#8230;</li>
<li>The <strong>macro menu</strong> is greatly improved, with hotkeys Ctrl-D for an improved menu and Ctrl-E for a quick-add menu.</li>
<li>We have a bona-fide <strong>game menu</strong> available from the F1 key. It&#8217;s almost like we&#8217;re a modern game! For now you can use it to access save, quit, macro, and help functions, but we&#8217;ll probably expand it in the future.</li>
<li>The <strong>allow_extended_colors</strong> console option is now on by default. For console nerds only! We previously didn&#8217;t enable it because some major system terminals didn&#8217;t support more than 8 standard colors by default, but that&#8217;s no longer the case.</li>
<li><strong>Hints mode</strong> has been updated and corrected. But it still recommends Vehumet over Sif Muna! Wrong!</li>
<li>The <strong>Desolation </strong>entrance is no longer <a href="https://github.com/crawl/crawl/commit/d28e58806775">ruined</a>.</li>
</ul>
<p>That wraps it up for this set of trunk updates. Until next time, happy crawling!</p>
<p><em>The damnation bolt hits the trunk update!!! The damnation bolt explodes! The damnation engulfs the trunk update! The trunk update dies&#8230;</em></p>


guid: http://crawl.develz.org/wordpress/?p=5029
title: 0.27 “The Cursed Flame”
author: gammafunk
url: https://crawl.develz.org/wordpress/0-27-the-cursed-flame
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We are pleased to announce the release of Dungeon Crawl: Stone Soup 0.27 “The Cursed Flame”!</p>
<p>DCSS 0.27 features a new Djinni species that cast spells using HP, can only learn new spells at random as they level up, and only use the spellcasting skill to train magic. We&#8217;ve redesigned Ashenzari, who now offers curses for your gear to gain piety and skill levels, at the cost of destroying the item when the curse is removed.</p>
<p>A batch of new spells have been added, including Manifold Assault, which attacks multiple foes at once with your melee, Storm Form, which transforms you into a tempest with electrified, cleaving melee and the ability to hurl yourself as lightning at foes, and Animate Armour, which draws out the spirit of your armor to fight by your side. Polar Vortex is a more damaging, ice-themed replacement for Tornado. Maxwell&#8217;s Capacitive Coupling is a level 8 air-themed replacement for Absolute Zero that vaporizes a nearby monster after a delay. Chain Lightning has been reworked into a more powerful level 9 spell.</p>
<p>Spell books have many new types and generate more often, but contain fewer spells. We&#8217;ve re-balanced branch monster sets and end maps, as well as made new end maps for Lair, Shoals, and Snake. As always, there are dozens of new deadly vaults, and many more changes small and large alike!</p>
<p>Download DCSS 0.27 <a title="download" href="http://crawl.develz.org/download.htm">here</a> or <a title="play online" href="http://crawl.develz.org/play.htm">play it online</a> on one of many servers across the world! Packages for Windows and OS X are available now. Linux debs are coming soon. See the full list of changes on the release page <a href="https://github.com/crawl/crawl/releases/tag/0.27.0">here</a>.</p>
<p>The tournament starts on <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210730T20">Friday July 30th 8pm UTC</a> and runs through <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210815T20">Sunday August 15th 8pm UTC</a>, with all online 0.27 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.27/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210806T20">Friday August 6th 8pm UTC</a>.</p>
<p>Many thanks to all those who have <a title="credits" href="https://raw.githubusercontent.com/crawl/crawl/master/crawl-ref/CREDITS.txt">contributed</a> to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.27 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.27 and a warm welcome to the first-time contributors, listed in italics. (Let us know if someone is missing, this data is tricky to collect in bulk!)</p>
<blockquote><p>4Hooves2Appendages, AdamPG, Aliscans, <em>Alexei Barnes</em>, Beargit, <em>Elliott Bernstein</em>,<em> dicedlemming</em>, dilly, <em>DreamDust</em>, Goratrix, hyperactiveChipmunk, Implojin, Alex Jurkiewicz,  <em>Zhang Kai</em>,  <em>l33t-d00d</em>, Nikolai Lavsky, <em>Christopher Landry</em>, MainiacJoe, mgdelmonte, <em>Mintice,</em> Andrew O&#8217;Neill, <em>Alexander Posch</em>, <em>robertxgray</em>, <em>Sastreii,</em> sdynet, <em>WhiteShark,</em> Shummie, Skrybe, Jeffery Stager, Cebolla Sunbeam, Wensley, <em>Mara Williamson</em></p></blockquote>


guid: http://crawl.develz.org/wordpress/?p=5084
title: 0.27 Tournament Results
author: gammafunk
url: https://crawl.develz.org/wordpress/0-27-tournament-results
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The <a title="0.27 tournament" href="https://crawl.develz.org/tournament/0.27/overview.html">0.27 tournament</a> is over. On behalf of the Dev Team, many thanks to all the server admins, outside contributors, bug reporters, and to the many DCSS players who made the 0.27 release and tournament possible! For 16 days, players could compete for tournament points and banners by playing 0.27 games on the public servers. This was the third tournament held after the major revision of the tournament rules done for 0.25. There were changes based on previous input, and once again we&#8217;ve collected the feedback posted on tavern and reddit.</p>
<p>Congratulations to <a href="https://crawl.develz.org/tournament/0.27/players/yermak.html">Yermak</a>, perennial champion, for once again taking first place in the <a href="https://crawl.develz.org/tournament/0.27/all-players-ranks.html">individual competition</a>, with 105,402 points! Over the course of the tournament, Yermak won 48 games, including a streak of 24 games (best in tournament), an 86M high score run (also best in tournament), an XL 14 win (also tied for best in tournament!), 27 Nemelex&#8217; Choice combos, 5 species high scores, 3 background high scores, and 19 tier 3 banners. In 2nd place was <a href="https://crawl.develz.org/tournament/0.27/players/sergey.html">Sergey</a> with 91,825 points and 34 wins, including a streak of 21 games, 3rd lowest turncount win, 2nd 15 win of the tournament, 4 species high scores, 3 background high scores, and 14 tier 3 banners. Rounding out the top 3 was <a href="https://crawl.develz.org/tournament/0.27/players/dilly.html">dilly</a> with 84,283 points points and 38 wins, including a streak of 7 wins, 28 Nemelex Choice wins (tied for most in tournament), and 18 tier 3 banners.</p>
<p>Our top three took several of the top spots in the individual category rankings, but they weren&#8217;t the only players on the leaderboard. <a href="https://crawl.develz.org/tournament/0.27/players/acrobat.html">Acrobat</a> set the top standard in <a href="https://crawl.develz.org/tournament/0.27/win-rate.html">Win Rate</a> once again, winning 37 out of 43 games over the tournament for an adjusted win rate of 84%. Third place finisher dilly tied with <a href="https://crawl.develz.org/tournament/0.27/players/prakerore.html">Prakerore</a> in the <a href="https://crawl.develz.org/tournament/0.27/nemelex-choice.html">Nemelex&#8217; Choice</a> competition, both winning 28 Nemelex combos for points over the course of the tournament. Yermak and Sergey took the number one and number two spots in the quest for <a href="https://crawl.develz.org/tournament/0.27/combo-high-scores.html">Combo High Scores</a>, with <a href="https://crawl.develz.org/tournament/0.27/players/cryingnoob.html">CryingNoob</a> rounding out the top 3. The <a href="https://crawl.develz.org/tournament/0.27/lowest-turncount-win.html">lowest turncount</a> was 9,903 turns, shared in a tie by <a href="https://crawl.develz.org/tournament/0.27/players/caminho.html">Caminho</a> and <a href="https://crawl.develz.org/tournament/0.27/players/drman.html">DrMan</a>. Seasoned speed runner <a href="https://crawl.develz.org/tournament/0.27/players/p0werm0de.html">p0werm0de</a> took the <a href="https://crawl.develz.org/tournament/0.27/fastest-real-time-win.html">fastest real-time win</a> with a 22:37 victory. Spriggan Enchanter was the power-combo of the<a href="https://crawl.develz.org/tournament/0.27/lowest-xl-win.html"> Low XL win</a> category, with Acrobat, Yermak, and Prakerore in a three way tie at XL14. The tournament start was heralded by p0werm0de who posted the first win after 33 minutes, and dilly brought home the first 15 rune win in an hour and 20 minutes. The <a href="https://crawl.develz.org/tournament/0.27/ziggurat-diving.html">Ziggurat Dive</a> was more restrained than the 0.26 tournaments&#8217; 100 ziggurat fiesta, with <a href="https://crawl.develz.org/tournament/0.27/players/ge0ff.html">Ge0ff</a> clearing 60 Ziggurats. Finally, both <a href="https://crawl.develz.org/tournament/0.27/players/poncheis.html">poncheis</a> and <a href="https://crawl.develz.org/tournament/0.27/players/booing.html">booing</a> earned <a href="https://crawl.develz.org/tournament/0.27/banner-collection.html">every banner</a>, the only two players to complete this feat.</p>
<p>The <a href="https://crawl.develz.org/tournament/0.26/teams.html">clan competition</a> was won by <a href="https://crawl.develz.org/tournament/0.27/clans/gozag-or-stay-home-ge0ff.html">Gozag or Stay Home</a> (captained by <a href="https://crawl.develz.org/tournament/0.27/players/ge0ff.html">Ge0ff</a>) with 105,343 points. In 2nd place was <a href="https://crawl.develz.org/tournament/0.27/clans/teamsplat-shummie.html">Team Splat</a> (captained by <a href="https://crawl.develz.org/tournament/0.27/players/shummie.html">shummie</a>) with 78,267 points. Following close on their heels in 3rd place were <a href="https://crawl.develz.org/tournament/0.27/clans/abcdef-fbynet.html">ABCDEF</a> (captained by <a href="https://crawl.develz.org/tournament/0.27/players/fbynet.html">fbynet</a>) with 76,046.</p>
<p>Most of the clan categories take the best score of the member, so I won&#8217;t recap those. The <a href="https://crawl.develz.org/tournament/0.26/clan-nemelex-choice.html">Nemelex&#8217; Choice</a> competition was a collective effort, and and just as in 0.26 it was quite high scoring. Gozag or Stay Home came out on top, with a total of 49 clan nemelex points; closely following were ABCDEF with 47. The <a href="https://crawl.develz.org/tournament/0.26/clan-combo-high-scores.html">Combo High Scores</a> competition was blown away by Gozag or Stay Home with 917 points in the category, a result of having both Yermak and Sergey on the team. This tournament only two clans <a href="https://crawl.develz.org/tournament/0.27/clan-banner-collection.html">collectively earned every banner</a>: ABCDEF thanks to the &#8216;B&#8217;, booing; and<a href="https://crawl.develz.org/tournament/0.27/clans/chatdocrawlbr-behavioral.html"> chatdotcrawlBR</a>, which had poncheis as a member.</p>
<p>Thank you to everyone for playing.</p>
<p>To close out, here are some overall tournament statistics (with the 0.26 numbers in brackets for comparison). 2,527 players started a game at some point during the tournament; of those 2,474 completed a game in a non-boring way. This definition of “player” (players who completed a non-boring game) is used for subsequent calculations and was used for the 0.27 statistics.</p>
<ul>
<li>Players: 2,474 (0.26: 2,333)</li>
<li>Total player time: 27,502 hrs (0.26: 29,392 hrs)</li>
<li>Average player time: 11.2 hrs (0.26: 12.6 hrs)</li>
<li>Games played: 67,203 (0.26: 54,701)</li>
<li>Players who got a rune: 734 (0.26: 886)</li>
<li>404 winners and 1571 wins (0.26: 476 winners and 1767 wins)</li>
<li>Win rate: 2.34% (0.26: 3.23%)</li>
<li>Proportion of players using WebTiles: 94.4% (0.25: 96.7%)</li>
</ul>
<p>The 0.28 season has begun, and it again looks to be a very active trunk season. Watch this space for further information as the 0.28 trunk update posts begin to flow. Until next time, happy crawling!</p>


guid: http://crawl.develz.org/wordpress/?p=5099
title: 0.27.1 Bugfix Release
author: gammafunk
url: https://crawl.develz.org/wordpress/0-27-1-bugfix-release
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We&#8217;ve uploaded the 0.27.1 bugfix release. Packages for Windows, MacOs, and Linux are all available on the <a href="https://crawl.develz.org/download.htm">download page</a>. Thanks to all the many contributors who helped with this release, including patches and bug reports!</p>
<p>Highlights (<a href="https://github.com/crawl/crawl/compare/0.27.0...0.27.1">full commit list</a> via github):</p>
<ul>
<li>The frequency of D:1 jackals is reduced.</li>
<li>Sigmund and Robin are nerfed gently.</li>
<li>Maxwell&#8217;s Capacitative Coupling now has LOS range.</li>
<li>80 other fixes, tweaks, copy-edits, and interface improvements.</li>
</ul>
<p>Updating is recommended for all players. Happy crawling, and report more bugs <a href="https://github.com/crawl/crawl/issues">here</a>!</p>


guid: http://crawl.develz.org/wordpress/?p=5112
title: Trunk Updates 29 August 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-29-august-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The trunk updates begin to smolder around you&#8230;</em></p>
<p>Hello crawlers, and welcome to the first trunk update of the 0.28 cycle. Coming fresh out of the gate are a <strong>new god Ignis</strong> and accompanying <strong>zealot background Cinder Acolyte</strong>, plus a big new feature for <strong>Okawaru</strong> and freedom from the <strong>summon XP penalty</strong>. On top of all that, a <strong>new unique Mlioglotl</strong> stalks the dungeon. Mlioglotl unique horror with the power to corrupt the level, changing terrain and summoning other abyssal horrors, as well as cause fear. He places in the S-branches and rarely in the Abyssal lair end. Thanks to community contributor <em>amcnicky</em> for the nightmares. Watch out!</p>
<h2>Ignis, the Dying Flame</h2>
<p>Ignis is a new god who does not appear in the temple. Indeed, altars to Ignis do not place in the regular dungeon at all! Ignis worship can only begin at character creation or from a faded altar. Ignis is dying, and as their last worshipper you are given all remaining power right from the get go.</p>
<ul>
<li>Ignis worshippers start at 5* piety (6* if a monk joining from a faded altar).</li>
<li>Ignis has no piety gain and no piety decay.</li>
<li>Once abandoned and mollified, Ignis dies and can never be worshipped again.</li>
<li>Ignis grants the following abilities:</li>
<ul>
<li>0*: Passive rF+</li>
<li>1*: Active &#8216;Sea of Flames&#8217; ability. Creates smoldering embers (like Conjure Flame) on every empty square in LOS. They ignite after the next player turn unless snuffed out by something occupying the cell.</li>
<li>1*: Active &#8216;Foxfire Swarm&#8217; ability. Conjures Foxfire on every open square in radus 2 around the player.</li>
<li>2*: Active &#8216;Rising Flame&#8217; ability. Gives the player the &#8220;Rise&#8221; status, after 2-4 turns the player rockets through the ceiling and up one floor.</li>
</ul>
</ul>
<p>The main way to worship Ignis is by starting as a <strong>Cinder Acolyte</strong>. Cinder Acolytes are a new background and start worshipping Ignis, along with: a 7/5/0 stat split, a robe, a -1 flame branded weapon of their choice, and the spell foxfire memorized.</p>
<h2>Okawaru, god of (single) Combat</h2>
<p>In the 0.27 cycle Okawaru gave up on their old honorable ally conduct. That was the beginning of a shift in their point of view on allies, and the god of combat now wants to see you prove yourself a worthy hero alone!</p>
<ul>
<li>Okawaru now <strong>forbids all allies</strong>, as if you had sacrificed love under Ru.</li>
<li>And gains a <strong>new 5* ability Duel</strong>. The Duel ability smite-targets a single non-summoned enemy in LOS (which must be threat level &#8220;tough&#8221; or higher), and transports them and the player to an Arena portal vault. Winning the duel opens a gate to return from the portal vault, and the player will be forcibly teleported back after a short duration.</li>
<li><strong>Finesse</strong> is moved to 4*.</li>
</ul>
<h2>Summoning Changes</h2>
<p>For a long time summons have applied an XP penalty, stealing up to half of the XP reward from each kill. This was used as a balance tool, because allies in crawl are very powerful. However, over the past many versions a lot of the sources of ally power (summons attacking out of LOS, sources of permanent, very long lived, or cheap and powerful allies) have been removed. Summoning spells were still extremely powerful, owing to the large number of creatures that could be brought to bear on a problem. In trunk summons and allies are changed in the following way, to enable removing the XP penalty.</p>
<ul>
<li>Summon caps are reduced:</li>
<ul>
<li><strong>Summon Small Mammal</strong>, <strong>Summon Hydra</strong>, and <strong>Monstrous Menagerie</strong> are capped at 2.</li>
<li><strong>Haunt</strong> is capped at 8.</li>
<li><strong>Summon Horrible Things</strong> and <strong>Dragon&#8217;s Call</strong> keep their old summon caps.</li>
<li><strong>All other summon spells</strong> are capped at 1.</li>
</ul>
<li>Summon-capped summons all time out uniformly and disappear after 10 auts.</li>
<li><strong>Shadow Creatures</strong> is made a monster only spell.</li>
<li>The <strong>XP penalty</strong> is removed from <strong>all temporary allies.</strong></li>
<li>Permanent allies from Beogh and Yredelemnul currently retain the XP penalty, with a hope to change their design in the future in a way that allows us to fully remove the XP penalty.</li>
</ul>
<h2>And all the rest</h2>
<p>Of course, in addition to big projects there are myriad tweaks, adjustments, and smaller features in trunk. Here&#8217;s the latest</p>
<ul>
<li><strong> Ziggurats</strong> no longer require a rune of zot to enter.</li>
<li>The<strong> Enchanter background</strong> no longer starts with the Corona spell, and instead starts with a potion of invisibility.</li>
<li>The<strong> evolution mutation</strong> is revised. Once per XL&#8217;s worth of experience it activates, providing a good mutation. Every two activations it reduces its level by one.</li>
<li><strong>New bad mutation: devolution</strong>. Mechanically similar to evolution but dispenses bad mutations and activates once every 1/4th of an XL.</li>
<li><strong>Strength 0</strong>&#8216;s regeneration reduction is replaced with -Regen in monster LOS.</li>
<li><strong>Intelligence 0</strong>&#8216;s 4/5 chance to fail scroll reading is replaced with -Scroll.</li>
<li>The <strong>staff of Battle</strong> now triggers the battlesphere on melee hit (in addition to the usual spell trigger).</li>
<li>All <strong>shields now only use Dex</strong> for stat scaling.</li>
<li>The <strong>robe of Clouds</strong>&#8216; thunder cloud ability is now passive, placing storm clouds on enemies within range 2 each turn.</li>
<li><strong>Riposte</strong> is removed from long blades and base damage is increased.</li>
<li>Demonspawn warmongers&#8217; <strong>Sap Magic</strong> effect now causes a short term -Cast after casting a spell.</li>
<li><strong>Lesser Beckoning&#8217;s</strong> power cap is reduced to 100 and range reduced to 5.</li>
</ul>
<h2>What&#8217;s on the horizon</h2>
<p>That&#8217;s all for now, crawlers, but there&#8217;s a lot cooking down here in Dis:7. There is a <a href="https://github.com/crawl/crawl/pull/2064">major refresh of the hells underway</a>, discuss play testing feedback on reddit or the tavern! Hellmonk has some hellish <em>new wizlabs</em> nearing completion. PleasingFungus has been at work in his own wizard laboratory and has two new spell designs that might make it to trunk soon. Beyond that, who can say what malicious whims will flit through the devteam&#8217;s head. So until next time, happy crawling!</p>
<p style="text-align: right;"><em>With a burst of heat and light, the trunk updates rocket away!</em></p>


guid: http://crawl.develz.org/wordpress/?p=5124
title: Trunk Updates 23 September 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-23-september-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>You feel a terrible foreboding&#8230; the trunk updates come into view!</em></p>
<p>Hello crawlers! It has been a busy month since we last wrote, with lots and lots of new stuff landing in trunk.</p>
<p>First of all, you&#8217;ve probably noticed a lot of fresh new looks for monsters and terrain. A big thanks to community contributor <em>Sastreii </em>for the stream of awesome tiles.</p>
<p>The long promised <strong>Hell overhaul</strong> has arrived in trunk, with a complete refresh of each of the sub-branches and tons of new monsters. <em>nicolae</em> is hard at work on new Hell vaults as well, so watch out! Speaking of vaults, hellmonk has brewed up <strong>six new WizLabs</strong> for you to loot, or perhaps to die in.</p>
<p><strong>Fighter</strong> and <strong>Fire Elementalist</strong> have both been adjusted &#8211; the latter has two new spells, and there are a few other new spells as well. The behavior of <strong>random rage</strong> from all sources has had a variety of tweaks, and there&#8217;s the usual grab bag of other tweaks, changes, and adjustments.</p>
<h2>Hell Overhaul:</h2>
<p>For a long time, players have let us know that hells are too long (28 floors!) but too easy &#8211; a &#8220;victory parade&#8221;. To fix that, we&#8217;ve taken what was fun and challenging about the hells and distilled away the rest. The result is much more lethal, shorter, and a rip-roaring good time all the way through. Changes include:</p>
<ul>
<li><strong>Hell effects</strong> are divided into timed effects and branch wide persistent effects.</li>
<li>The <strong>timed effects</strong> are the same for all hells, and consist of temporary mutation, stat drain, hp drain, and noise.</li>
<li>The <strong>branch-wide persistent effects</strong> are:</li>
<ul>
<li>In Dis, players have 8 Corrosion (-8 AC and Slaying).</li>
<li>In Cocytus, potions are frozen solid cannot be used.</li>
<li>In Gehenna, scrolls are obscured by hellish smoke and cannot be used.</li>
<li>In Tartarus, Willpower is halved by the sheer horror of the environs.</li>
</ul>
<li>In each hell sub-branch, floors 1-6 are shrunk to 1/4 of their original area. (The final floor of each branch, floor 7, is unchanged.)</li>
<li>Each level only generates a single downstair, with no randomly placed shafts and a limited number of escape hatches.</li>
<li>Hell subbranches have their monster lists refreshed with a variety of new enemies, replacing things like weak derived undead and so-called &#8220;trash mobs&#8221;.</li>
<ul>
<li>Each subbranch has hellions, tormentors, dread liches, and ancient liches.</li>
<li>In addition to the cross-hell population, there are revised monster sets in each subbranch.</li>
<li><strong>Gehenna</strong> is now populated by brimstone fiends, hellephants, hell hogs, fire giants, salamander tyrants, balrugs, and three new monsters:</li>
<ul>
<li>The <strong>creeping inferno</strong> stalks the player like a lurking horror, and when it explodes it goes off in a fire storm.</li>
<li><strong>Searing wretches</strong> are ghoul-like monsters that strip fire resistance with their melee attacks.</li>
<li><strong>Stokers</strong> tend the fires of Gehenna, summoning creeping infernos and setting off wind blasts.</li>
</ul>
<li><strong>Dis</strong> is now populated by hell sentinels, iron giants, caustic shrikes, iron dragons, war gargoyles, buffed iron golems, and two new monsters:</li>
<ul>
<li><strong>Quicksilver elementals</strong> have quicksilver bolt and antimagic attacks, as well as batty movement.</li>
<li><strong>Crystal echidnas</strong> throw barbs and fire crystal bolts. They&#8217;re made of green crystal and are appropriately vulnerable to LRD.</li>
</ul>
<li><strong>Cocytus</strong> is now populated by ice fiends, shard shrikes, azure jellies, titans, frost giants, freezing wraiths, simulacra, and two new monsters:</li>
<ul>
<li><strong>Wendigos</strong> have stunning blast, paralyzing victims for 1 turn on hit. They can also turn nearby simulacra into very powerful iceblasts, destroying the simulacrae in the process.</li>
<li><strong>Narguns</strong> hit hard, have a vulnerability melee, and cast petrify. (Watch out &#8211; since potions are frozen solid in Cocytus, you can&#8217;t quaff cancellation to stop the petrification!).</li>
</ul>
<li><strong>Tartarus</strong> is now populated by tzitzimimeh, silent spectres, doom hounds, bone dragons, profane servitors, eidola, shadow wraiths, spectral things, and two new monsters:</li>
<ul>
<li><strong>Tainted leviathans</strong> are undead giants which mesmerise, throw large rocks, and are ringed by miasma clouds</li>
<li><strong>Putrid mouths</strong> are screaming ghosts with miasma breath, strong poison and stat draining melee.</li>
</ul>
</ul>
</ul>
<h2>Six New Wizard&#8217;s Laboratories:</h2>
<p>Hellmonk, with an assist from PleasingFungus and art from Sastreii has brought a whole assortment of new WizLabs for you, including restoring two that were cut in 0.17.</p>
<ul>
<li><strong>Eringya&#8217;s Formal Garden</strong> (previously removed in 0.17) returns with a modified layout, nastier archers, and the addition of some of the swamp monsters that have been added between then and now.</li>
<li><strong>Tukima&#8217;s Studio</strong> (also removed in 0.17) returns with a modified layout and several brand-new monster types to complement the dancing weapons: walking tomes, which summon living spells that cast themselves at the player. Walking Tomes also now appear as high-tier enemies in Depths.</li>
<li><strong>Yara&#8217;s Duelist Academy</strong> features mana vipers, lots of warrior-mages wielding antimagic weapons, and a fearsome semi-unique Head Instructor who likes to dual-wield triple swords.</li>
<li><strong>Borgnjor&#8217;s Mausoleum</strong> features both corporeal and incorporeal undead. Lost souls occasionally appear when monsters are killed, making big fights scarier. A cult of halazid warlocks serve as the final encounter.</li>
<li><strong>Maxwell&#8217;s Workshop</strong> features an array of golems and big, meaty workers who wield big, meaty maces. You&#8217;ll find some of Maxwell&#8217;s cast-off equipment here, perhaps including the legendary artifacts themselves.</li>
<li><strong>Alistair&#8217;s Party Mansion</strong> features an array of wizards and their bizarre guests. There&#8217;s some good consumables to be had if you don&#8217;t mind crashing the party of the century&#8230; and wading through statrotting clouds of degeneration.</li>
</ul>
<div>
<h2>Background changes:</h2>
<p>In 0.27, we removed some Level 5 spells from book starts. This was to push them toward being like other backgrounds: a starting kit, not something that lasts you through most of the game. Fire Elementalists are now being changed in the same way: they get two new low-level spells, Scorch and Flame Wave (described below), replacing Sticky Flame and Fireball in the starting book. </p>
<p>Changes to shield delays made kite shields a worse starting item. Accordingly, most Fighters now start with bucklers, which require less skill. (Ogres and Trolls, which cannot use bucklers, are unchanged.)
</p></div>
<h2>Spell changes:</h2>
<ul>
<li><strong>New spell:</strong> Scorch (L2 Fire). Hits a random enemy within 3 tiles, dealing fire damage. If damage is dealt, applyies short term rF- to the foe. <strong>Cinder Acolytes</strong> start with Scorch instead of Foxfire.</li>
<li><strong>New spell:</strong> Flame Wave (L4 Fire/Conj). Flame Wave is a channeled spell (like Searing Ray) that hits all surrounding tiles in ever-wider waves of flame. The initial cast costs 4 MP, and each turn of channeling costs 1 MP. The noise starts a bit louder than a shout, and gets up to fireball level by the last wave.</li>
<li><strong>New spell:</strong> Summon Cactus Giant (L6 Summoning). Summons a jolly green giant, which impales attacking enemies on its giant spines. Appears in the Book of the Wilderness and the Book of Scorching</li>
<li><strong>Searing Ray</strong>&#8216;s power scaling is decreased.</li>
<li><strong>Storm Form&#8217;s</strong> power scaling is decreased, but blinkbolt is now always LOS range.</li>
<li><strong>Bolt of Magma</strong> and<strong> Corona</strong> are now monster only.</li>
</ul>
<div>
<h2>Random rage:</h2>
<ul>
<li><strong>Random rage</strong> bypasses Ashenzari clarity</li>
<li><strong>*Rage</strong> now causes berserk on 20% of hits per item (the exact chance is displayed in the % screen and item description).</li>
<li>The <strong>evokable berserk</strong> artefact property is removed.</li>
<li>Unrand changes:</li>
<ul>
<li>The<strong> necklace of bloodlust</strong> keeps *Rage but with 50% berserk chance.</li>
<li>The <strong>Zealot&#8217;s sword</strong> retains *Rage with the standard 20% berserk chance.</li>
<li><strong>Leech</strong> trades *Rage for Harm and Drain on unequip.</li>
<li>The <strong>hat of the bear spirit</strong> gains *Rage instead of +Rage (for now).</li>
</ul>
<li><strong>Berserkitis&#8217;</strong> chance to berserk on melee hit stacks with *Rage linearly, and the cumulative chance is displayed in the % screen.</li>
<li><strong>Cheibriados</strong> permits berserk, and just suppresses the extra speed.</li>
</ul>
</div>
<h2>The rest of the changes:</h2>
<ul>
<li>All <strong>derived undead</strong> keep their reach, constriction, and trample effects.</li>
<li>New entry vaults, ruin sub vaults, and maps for <strong>Desolation</strong></li>
<li><strong>Red Devils</strong> no longer kite, and instead their melee attack inflicts the barbs status if damage is done.</li>
<li>The <strong>Ranger Card</strong> now summons rangers from these three types: centaur warrior, naga sharpshooter, deep elf master archer. (thanks <em>sdynet!</em>).</li>
<li><strong>Ironbound Beastmasters</strong> are removed.</li>
<li><strong>Depths</strong> is now 4 floors.</li>
<li>The <strong>experience cost of high-level skills</strong> is reduced (approximately equivalent to a +1 increase in aptitude at skill level 23).</li>
<li>To compensate, <strong>late game XP is reduced</strong>.</li>
<li>The <strong>top-end two handers</strong> have slightly lowered skill requirements for mindelay. Executioner&#8217;s axes and bardiches now reach mindelay at skill 24, and triple swords at skill 22.</li>
<li><strong>Throwing ammunition</strong> is reduced.</li>
<li><strong>Abyssal Knights</strong> background stat contribution is changed to 5/2/5 Str/Int/Dex, from 4/4/4.</li>
</ul>
<h2>On the horizon:</h2>
<p>That rounds out the changes for this update. Looking to the future is always fraught, but amid the miasmatic haze of Tartarus there are some outlines of a <em>spider overhaul</em>, <em>adjustments to the wanderer start</em>, <em>more efforts to compactify mid-to-lategame</em>, and a host of community and dev-team designed <em>new vaults. </em>Further afield there are rumors of new Bailies, overhauls of Yredelemnul and Jiyva, and perhaps even something about ranged combat. Stay tuned! And until next time, happy crawling!</p>
<p style="text-align: right;"><em>The trunk update rustles loudly! The trunk update casts itself at you!!!!!!! You die&#8230;</em></p>


guid: http://crawl.develz.org/wordpress/?p=5140
title: Trunk Updates 6 December 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-6-december-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The blog shudders from the hit and a flood of trunk updates pours out!</em></p>
<p>Hello crawlers! Sorry for not writing, there was a leak in the lava system down here in Gehenna and it was all hands on deck for the Devil&#8217;s team to get it contained. Despite the chaos, since the last trunk update there have been a whopping 480 commits to trunk. So buckle up for a meaty post. Before we dive into the details, some headlining information.</p>
<p>First, a warm welcome to <em>Sastreii</em>, who has joined the dev team as a tiles artist and has been prolific in refreshing old tiles and filling in art for the new content that&#8217;s arrived in this cycle.</p>
<p>Next, on the horizon in this version are <strong><a href="https://github.com/crawl/crawl/pull/2150">a refresh of the Spider branch</a></strong> and an <strong><a href="https://github.com/crawl/crawl/pull/2267">overhaul of Yredelemnul</a></strong>. Both are currently playable on the experimental <a href="http://crawl.dcss.io">crawl.dcss.io</a> development server (not tracked in scoring or Sequell). Playtesting feedback is welcome and wanted on these branches, drop us a line on GitHub, reddit, the tavern, or #crawl-dev on libera.chat if you have <em>substantive feedback based on playing experience</em>. Both sets of changes should arrive in trunk very soon, but they aren&#8217;t quite there yet.</p>
<p>Also on the immediate horizon: a combined effort by community contributor <em>nlavsky</em> and team member <em>ad</em><em>vil </em>will soon deliver a <strong>consumables panel for </strong><strong>web tiles</strong>. This has been a desired feature for a long time, so hats off to both of them for the effort!</p>
<p>A big thanks to community contributors <em>ManiacJoe</em> and <em>DreamDust</em> for new vaults, as well as to <em>AdamPG, Aliscans, ArmiesAndCastles, Brian Power, David Damerell, Gopall, Implojin, Martin Kennedy, nlavsky, perryprog, Santiago Acosta, dilly, mgdelmonte, paul,</em> and <em>robertxgray </em>for patches with tweaks, bug fixes, balance adjustments, lua improvements, and more since we last wrote.</p>
<p>With all that said, lets dive into the trunk changes!</p>
<h2>Branches &amp; Environment</h2>
<ul>
<li>Exploration-based <strong>shaft traps</strong> are now limited to one shaft per branch per game.</li>
<li>Portal vault <strong>portal mimics</strong> have cackled and vanished for the last time, and are removed.</li>
</ul>
<h2>Character</h2>
<ul>
<li>Two new <strong>Demonspawn </strong>mutations:</li>
<ul>
<li>The <strong>weakening stinger</strong> mutation gives a tail aux attack. At level 3 it blocks the cloak slot and the aux attack applies the weak status.</li>
<li>The <strong>demonic touch</strong> mutation adds irresistible damage to the off-hand attack. At level 3 it blocks the gloves slot and applies the vulnerable status.</li>
</ul>
<li><strong>Player-chosen attribute increases</strong> now happen every 6 XLs, starting at XL 3, and increase a stat by 2. Demigods are unchanged.</li>
<li><strong>Black Draconian Breath</strong> no longer includes shock.</li>
<li><strong>Clarity </strong>from any source again blocks <strong>*Rage</strong>.</li>
</ul>
<h2>Gods</h2>
<ul>
<li><strong>Ashenzari&#8217;s </strong>passive <strong>trap avoidance</strong> and <strong>item identification</strong> are moved 1*. No more freebies without a curse!</li>
<li><strong>Cheibriados&#8217;</strong> passive <strong>poison slowing</strong> is moved to 0* and <strong>bend time</strong> moved to 1*.</li>
<li><strong>Elyvilon&#8217;s </strong>abilities are streamlined to be less redundant with one another and with Zin&#8217;s sanctuary:</li>
<ul>
<li><strong>Pacification</strong> grants full XP, and can be used on intelligent plants.</li>
<li><strong>Lifesaving</strong> is completely passive, and is a bonus chance on top of the usual good god chance that becomes guaranteed at 5*, but with a piety-gated timeout.</li>
<li><strong>Lesser Healing </strong>is removed, with <strong>Greater Healing</strong> renamed to <strong>Heal Self</strong> and moved to 3*.</li>
<li><strong>Purification</strong> is moved to 1* and is cheaper.</li>
</ul>
<li><strong>Gozag&#8217;s</strong> gold distraction chance is capped.</li>
<li><strong>Ignis&#8217; </strong>abilities are shifted based on playtesting experience:</li>
<ul>
<li><strong>Rising Flame</strong> is now a one-time use ability, but costs no piety.</li>
<li>The cost of the other two active abilities is increased to compensate.</li>
<li>The <strong>Sea of Flames</strong> ability is removed and replaced with <strong>Fiery Armour</strong>, a long lasting duration that gives +7AC and causes melee attackers to suffer fire damage.</li>
</ul>
<li><strong>Jiyva</strong> gurgles merrily and shifts into a new form:</li>
<ul>
<li><strong>Stat shuffling</strong>, <strong>healing and piety from item eating</strong>, and <strong>cure mutation</strong>, are all removed.</li>
<li>Piety is now <strong>exploration based</strong> and grows very quickly.</li>
<li>Jelly healing is replaced with <strong>passive HP and MP regeneration</strong>.</li>
<li><strong>Mutation gifts</strong> arrive in bursts that attempt to remove a mutation (more likely a bad one) and then max out a good one.</li>
<li><strong>rCorr</strong> is now granted at 2*, and <strong>jelly-on-hit</strong> at 5*.</li>
<li>New active ability <strong>Oozemancy</strong> granted at 3*. Somewhat expensive in piety it temporarily turns walls in radius 4 into slime walls.</li>
</ul>
<li>The chance of <strong>Ru</strong> retaliation is reduced.</li>
<li><strong>The Shining One</strong> abilities arrive earlier, with <strong>Divine Shield </strong>at 1* and <strong>Cleansing Flame</strong> at 3*.</li>
</ul>
<h2>Items</h2>
<ul>
<li>The spawn rate of <strong>wands</strong> is reduced.</li>
<li>The <strong>electrocution </strong>brand now triggers on fewer attacks.</li>
<li><strong>Regen</strong> is reduced to 0.8 HP / turn per pip.</li>
<li><strong>MPRegen</strong> is increased to 0.4 MP / turn.</li>
<li>The <strong>brand weapon chances</strong> have been tweaked, and are now described in the scroll&#8217;s description.</li>
<li>The spawn rates of <strong>scrolls and potions</strong> are adjusted slightly and now described in in-game descriptions.</li>
<li>New glove ego: <strong>infusion</strong>. Gloves of infusion add damage per hit in exchange for 2 MP per hit.</li>
<li>New unrandart: the <strong>+3 Mad Mage&#8217;s Maulers {Infuse+∞ RegenMP Int-3}</strong>. These exchange all of the wearer&#8217;s current MP in exchange for bonus damage proportional to the amount of MP used.</li>
<li>The <strong>hat of the Bear Spirit</strong> now grants extra bonus HP while berserk (for a total of +100% HP) and reduces the post-berserk slow duration, though it loses rN+.</li>
<li>Unrandarts that <strong>produce allies</strong> no longer do so when worshipping Okawaru.</li>
</ul>
<h2>Monsters</h2>
<ul>
<li><strong>Demonic</strong> and <strong>Holy</strong> monsters are now susceptible to fear, intoxication, and mushroom spores, and are able to go berserk.</li>
<li>Monsters now have a chance to <strong>break a door permanently</strong> when opening it to get to the player. This chance is higher if berserk.</li>
<li><strong>Classed </strong><strong>draconians </strong>now come in a fixed colour based on their job.</li>
<li><strong>Plain demonspawn</strong> are removed. Each class of classed demonspawn has one mix of demonspawn attributes.</li>
<li>When <strong>Dowan </strong>dies, <strong>Duvessa</strong> goes berserk permanently; when roles are reversed Dowan gains haste permanently.</li>
<li>Merfolk <strong>impalers</strong> and <strong>javlineers</strong> are nerfed slightly.</li>
<li><strong>Walking Tomes</strong> have a summon cap on their living spells.</li>
<li><strong>Shard shrikes</strong> now engage in batty wandering after casting icicle once, as they were meant to.</li>
<li>On death, <strong>Eleionoma</strong> and <strong>Fenstrider Witches</strong> drop a wand if they drop a corpse (similar to a dragon&#8217;s hide).</li>
</ul>
<h2>Spells</h2>
<p>Within the spell changes, a big chunk of them involve making existing spells scale with power and making sure the power caps for low level spells aren&#8217;t too high. Here&#8217;s that set first:</p>
<ul>
<li><strong>Corpse Rot</strong> scales with power, placing more miasma clouds per corpse and placing clouds at radius 2 if all the radius 1 space is already filled.</li>
<li><strong>Passage of Golubria</strong> now has a range that scales with power.</li>
<li><strong>Blink </strong>has a cool down that scales (downwardly) with power.</li>
<li><strong>Teleport Other</strong>, <strong>Sublimation of Blood</strong>, <strong>Spider Form, Inner Flame</strong>, <strong>Gell&#8217;s Gravitas,</strong> and <strong>Vampiric Draining</strong> all have their power caps reduced.</li>
</ul>
<p>And the rest of the spell changes:</p>
<div>
<ul>
<li><strong>Eringya&#8217;s Noxious Bog</strong> now places a bog under the caster, as well as on cells with only one adjacent wall; the caster is now immune to their own bog.</li>
<li><strong>Inner Flame</strong> is now smite targeted.</li>
<li><strong>Slow</strong> is now level 1 with a lower power cap.</li>
<li>When <strong>Metabolic Englaciation</strong> successfully slows a monster it does so for a minimum duration.</li>
<li><strong>Death Channel</strong> now works on demonic and holy monsters, as well as corpseless living monsters.</li>
<li><strong>Flame Wave</strong> does less damage per wave.</li>
<li>Player <strong>hexes</strong> other than Ensorcelled hibernation now set allies to attack the hex target.</li>
<li><strong>Dragon Form&#8217;s </strong>breath weapon is buffed in power for non-draconians (to the power level of draconians).</li>
</ul>
<p>Wow. That was quite a lot, with lots of big projects on the horizon as well. So stay tuned! Remember to check out the experimental branches and give feedback. As always, if you are interested in contributing to the community effort <a href="https://github.com/crawl/crawl#how-you-can-help">head on over to our GitHub.</a> Until next time, happy crawling!</p>
<p style="text-align: right;"><em>Residual magic twists the trunk update into a wand of ending (1).</em></p>
</div>


guid: http://crawl.develz.org/wordpress/?p=5146
title: Trunk Updates 18 December 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-18-december-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>You begin with the following items: a +0 vorpal quarterstaff, a piece from Xom&#8217;s chessboard (15/15), a +0 cloak, a +0 leather armour, and a trunk update&#8230;</em></p>
<p>Hello crawlers! We&#8217;re getting back to a more regular trunk updates post schedule, but this comes hand in hand with a continued feverish pace of change. Since we last wrote there have been 186 commits to trunk. Unlike last time a much larger fraction of these are fixes, tweaks, and technical behind the scenes work that enable the headliner items.</p>
<p>First, <em>a big congratulations and thank you </em>to longtime community contributor <em>nlavsky </em>and team member <em>advil</em> for initiating and seeing through to completion the implementation of a <strong>consumable panel for </strong><strong>web tiles, </strong>similar to the inventory pane of local tiles. This has been an outstanding feature request for almost a decade, and is the number one reason offline players cite for why they don&#8217;t play online. So if you&#8217;re a long-time offline player and you don&#8217;t mind playing trunk, come check it out; otherwise, we look forward to you trying it during the 0.28 release! In addition to this panel, <em>advil</em> has been doing a lot of work <strong>expanding the web tiles mouse support</strong>, the second most requested online feature. Try it out!</p>
<p>Since we last wrote, we also had community contributors <em>Aliscans, ArmiesAndCastles, Brian Power, EdDewey, Gopall, Goratrix, Jonathan Klabunde Tomer, Oneirical, Perry Fraser, aMcNicky, mgdelmonte, Zachary Chandler, </em>and <em>sdynet</em> send in various tweaks, fixes, new items, lua api improvements (<em>mgdelmonte</em> is a big driver of these), and other assorted changes. As always, a big thank you to the community for participating in development.</p>
<p>On the gameplay side, the two big headline changes are the <strong>Yredelemnul overhaul</strong> and a revision of the <strong>Wanderer</strong> background start. Let&#8217;s dive in.</p>
<h2>Carry the Black Torch!</h2>
<p dir="auto">Last post I mentioned an experimental overhaul of Yredelemnul looking for playtesting feedback. This overhaul has arrived in trunk, after a set of balance tweaks and bug fixes thanks to your feedback. Here&#8217;s what Yredelemnul now offers, instead of permanent ally gifts.</p>
<ul dir="auto">
<li>At 0* Yredelemnul grants <strong>passive reaping</strong> of living and demonic creatures, as well as <strong>recall,</strong> and immunity to umbra and spectral mist clouds. Things with a corpse come up as zombies, things without as spectrals (including those poor souls whose corpse was exploded). These reaped creatures stick around until the player leaves the level.</li>
<li><strong>Piety</strong> is proportional to the total strength of reaped dead in the player&#8217;s LOS.</li>
<li>At 2* grants the <strong>Dark Bargain</strong> active, which trades some zombies/specters for a long-duration undead servant summon. Summon quality scales with piety, and the summon list now covers all (non-harmful) ghoul, skeleton, vampire, and wraith genus monsters (except Vampire magi who belong to Kiku); for console players that&#8217;s monsters on glyphs n, z, V, and W.</li>
<li>At 4* grants <strong>Drain Life</strong>. Same as old drain life, but now costs zombies/specters as well as mp.</li>
<li>At 5* grants <strong>Bind Soul</strong>. Same as old enslave soul, but also costs zombies and has been renamed. Bound souls can follow the player between levels, retain the spells and weapons they had when alive, and are tougher than before.</li>
</ul>
<h2>Wandering About</h2>
<p><em>hellmonk </em>took the charge in re-designing the random generation process of the Wanderer background. I won&#8217;t bore you with the old creation model, Wanderer players will know that the bad rolls could just be absolutely nonsensically awful. The new generation model picks three skills in an apt-weighted way: a &#8220;good&#8221; skill and two &#8220;decent&#8221; skills. Using these choices it hands out starting gear, &#8220;good&#8221; stuff for the good skill and &#8220;decent&#8221; stuff for the decent skill. Finally it assigns stats and skill points based on these choices, again in an apt-weighted way. The result is more lopsided and more likely to align with the gifted items, though it is not guaranteed to be that way. Along with a change to the random model, the possible items have been expanded, with some carried over from the old model. To entice you to check it out, here&#8217;s the possible list:</p>
<ul>
<li>&#8220;good fighting&#8221; selects a weapon and gives the good weapon instead. &#8220;decent fighting&#8221; gives an extra plus to a weapon if one was already given, or a decent potion or scroll otherwise.</li>
<li>&#8220;good weapon&#8221; gives a +2 weapon, an upgraded base type or a vorpal branded starting weapon (hunting slings with sling bullets are now possible). &#8220;decent weapon&#8221; just gives a plain weapon.</li>
<li>&#8220;good unarmed&#8221; gives two random good potions or scrolls. &#8220;decent unarmed&#8221; gives a single decent potion or scroll.</li>
<li>&#8220;good armour&#8221; gives either +2 scale, chain, or acid scales to weirdly shaped wanderers. &#8220;decent armour&#8221; gives either plain scale or ring mail, or aux armour to weirdly shaped wanderers.</li>
<li>&#8220;good shield&#8221; can give either a +2 buckler or a +0 buckler of protection (upgraded to a kite shield for species that can&#8217;t use a buckler). &#8220;decent shield&#8221; gives a +0 buckler.</li>
<li>&#8220;good magic school&#8221; gives a 3-spell randbook (capped at level 4) from that school. &#8220;decent magic school&#8221; gives a 2 spell book totaling 4 levels from that school.</li>
<li>&#8220;good spellcasting&#8221; gives a 4-spell randbook (capped at level 3), picking spells from any school. &#8220;decent spellcasting&#8221; gives a 2 spell book totaling 4 levels from any school.</li>
<li>&#8220;good stealth&#8221; gives either a +2 dagger and a good consumable or a +0 dagger and atropa and datura darts. &#8220;decent stealth&#8221; gives a piece of aux armour.</li>
<li>&#8220;good throwing&#8221; gives a +2 dagger and curare darts. &#8220;decent throwing&#8221; gives either boomerangs or poison darts.</li>
<li>&#8220;good dodging&#8221; gives either a +2 leather armour or a +0 leather armour and some nets (oddly shaped wanderers get an aux armour instead of leather). &#8220;decent dodging&#8221; gives a piece of aux armour.</li>
<li>&#8220;good evocations&#8221; gives either a wand (weighted towards the stronger wands) with a good number charges or an XP evoker other than the chess piece. &#8220;decent evocations&#8221; gives a handful of flame or polymorph charges or a piece from Xom&#8217;s chessboard.</li>
<li>The possible range of starting consumables is expanded.</li>
<li>If starting with the sandblast spell, a wanderer also receives stones.</li>
</ul>
<h2>Trunk Updates</h2>
<p>Of course, with 184 commits, even with all of the User Interface work there are plenty of other gameplay changes around. Read on for the mix</p>
<ul>
<li><strong>New Unrand: the dreamshard necklace</strong>. Concept and initial implementation by community contributor <em>aMcNicky</em>, the dreamshard necklace is an amulet an amulet of nothing that for the most part does nothing. However, on a lethal or near-leathal blow the necklace shatters, absorbing the blow and releasing protective dreams of healing (a big heal, between half and full HP), friendship (shadow creatures), and clouds (fog).</li>
<li>The <strong>ring of the Mage</strong> loses Will++ and Int+3 and trades them for <strong>Archmagi</strong>. Multiple sources of Archmagi now stack, so have fun blasting!</li>
<li>The <strong>autumn katana</strong> loses Clarity and trades it for a chance to cast <strong>Manifold Assault</strong> on hit, slicing through space because it is too sharp. This unrand refresh by community contributor <em>sdynet</em>.</li>
<li>The monster mix in <strong>Dis</strong> and <strong>Cocytus</strong> is adjusted to bring them closer in line with Gehenna and Tartarus threat levels.</li>
<li>Okawaru&#8217;s <strong>Duel</strong> ability is more expensive, and the post-duel grace period shorter.</li>
<li><strong>Quazlal</strong> abilities have had their costs balanced: <strong>Upheaval</strong> is cheaper in MP and Piety, <strong>Elemental Force</strong> cheaper in MP, and <strong>Disaster Area</strong> more expensive in MP.</li>
<li><strong>Chain Lightning </strong>damage now falls off more quickly with range.</li>
<li><strong>Yara&#8217;s Violent Unravelling</strong> can now unravel summons in addition to enchantments. When targeting a summon it causes the summon to explode, killing it instantly. Additionally, it is now <strong>smite targeted</strong>.</li>
<li>The <strong>briar patch</strong> monster now blocks all non-aligned beams, shielding its allies from all hostile fire, while allowing allied beams to fire through them. <strong>Bushes</strong> allow all beams to pass through, regardless of alignment.</li>
<li>The <strong>infusion</strong> ego is buffed, now costing only one MP per hit.</li>
</ul>
<p>There&#8217;s still a lot more brewing on the horizon this trunk season, with a Spider refresh and a lot of new and updated vaults on the way. So until next time, happy crawling!</p>
<p style="text-align: right;"><em>You go downstairs. The trunk update is left behind to rot!</em></p>


guid: http://crawl.develz.org/wordpress/?p=5152
title: Trunk Updates 28 December 2021
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-28-december-2021
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The broodmother hits you!!! A trunk update bursts forth from the wound!</em></p>
<p>A ho ho ho from Santaeus here in the icy expanse of <del>Cocytus</del> the North Pole! Welcome to the last trunk update of 2021. The frenzied pace of trunk development continues, with 150 commits since we last wrote. One big chunk of commits was some behind the scenes web tiles modernization, with some help from CKO admin and community contributor <em>floraline. </em>There are three big headliner sets of changes: a <strong>new set of Spider monsters</strong>, a variety of <strong>new early-game appearing </strong><strong>unrands</strong>, and a <strong>new type of item to carry in your offhand instead of a shield</strong>. Before we get to those three change sets, let&#8217;s check out the rest of the trunk changes:</p>
<ul>
<li>The <strong>glaive of Prune</strong> now turns you into a prune when paralysed, petrified, or put to sleep, causing you to take half damage until the effect ends.</li>
<li>If Jiyva dies <strong>Dissolution</strong>&#8216;s summon eyeballs is replaced with a cantrip.</li>
<li><strong>Sewer</strong> maps have been updated and re-vamped, plus two new sewer maps.</li>
<li><strong>Bailey</strong> maps have been updated and re-vamped, plus three new baileys.</li>
<li><strong>New Spell: Enfeeble</strong> (L7 Hexes). Enfeeble irresistibly applies weakness and antimagic to a single foe. Additionally the target is also dazed and blinded if they fail a willpower check.</li>
<li><strong>Formicids</strong> now <strong>dig</strong> at normal movement speed, and each cell dug causes draining.</li>
<li><strong>Centaur warriors</strong> replace centaurs in Shoals.</li>
</ul>
<h2>New and updated Spider monsters</h2>
<ul>
<li><strong>Pharaoh ants</strong> are an undead spider that drains and when killed binds the souls of all nearby monsters.</li>
<li><strong>Steelbarb</strong> worms are fast, regenerative, fire-resistant worms that apply barbs in melee.</li>
<li><strong>Jorogumo</strong> are tough ensnaring monsters with bolt of draining and hornet poison melee.</li>
<li><strong>Broodmothers</strong> summon spiders and bite to summon spiders.</li>
<li><strong>Culicivora</strong> are healing spiders, with heal other and a vampiric bite, appearing in spider packs.</li>
<li><strong>Sun moths</strong> are holy moths with a single tile halo, sticky flame, and energy bolt.</li>
<li><strong>Radroaches</strong> are armoured roaches that cast irradiate, which contaminates the player a lot (not a direct malmutation).</li>
<li><strong>Demonic crawlers</strong> gain warning cry.</li>
<li><strong>Emperor scorpions</strong> are less tough but faster.</li>
<li><strong>Ghost moths</strong> no longer poison but their stat draining is much more intense.</li>
</ul>
<h2>Early Game Artefacts</h2>
<p><span style="font-size: 13px; font-weight: normal;">Artefacts can now generate on D:2-3. Additionally, s</span>everal new unrandarts specifically designed to be interesting early and then fall off are added:</p>
<ul>
<li><strong>Delatra&#8217;s gloves</strong> heal the wearer for drinking unknown potions and restore MP for reading unknown scrolls.</li>
<li>The <strong>woodcutter&#8217;s axe</strong> is a vorpal +8 war axe that always swings at 10 aut, regardless of skill, shield penalty, slow or haste, etc.</li>
<li>The <strong>staff of the Meek</strong> is a +7 quarterstaff of protection with an extra protective bonus AC when below 55 HP.</li>
<li><strong>Throatcutter</strong> is a +7 long sword of draining with a 1/3 chance to instantly kill foes that are below 21 HP.</li>
</ul>
<div>Along with the new early-game unrandarts, several existing unrandarts have been given &#8220;early-game&#8221; status and weighted to be very likely early and unlikely later: <strong>Morg</strong>, <strong>Devastator</strong>, the <strong>Ratskin Cloak</strong>, <strong>Kryia&#8217;s mail coat</strong>, the trident of the <strong>Octopus King</strong>, the robe of <strong>Augmentation</strong>, and <strong>Lear&#8217;s Hauberk</strong> (reduced to +18).</div>
<h2>Orbs to ponder</h2>
<p>This round of updates brings a new item type: orbs. A shield-slot analog of scarves that give no SH and no penalty. Several new armour egos are added that appear only on orbs:</p>
<ul>
<li><strong>light</strong>: a radius 3 halo.</li>
<li><strong>wrath</strong>: gives *Rage (20% berserk chance).</li>
<li><strong>mayhem</strong>: when the wearer kills something, it has a chance to send a monster that saw the kill into a frenzy.</li>
<li><strong>guile</strong>: reduces the will of the wearer&#8217;s target by 3 pips, while reducing the wearer&#8217;s will by 2 pips.</li>
<li><strong>energy</strong>: gives a source of passive channeling: a chance to power a spell instead of using the wearer&#8217;s MP, with a chance to backfire and drain int or cause confusion. The success chance goes up with each source of passive channeling, and the backfire effect triggers if each source backfires. The <strong>staff of Wucad Mu</strong> now counts as two sources of channeling (so it stacks with an orb of energy in addition to being buffed!).</li>
</ul>
<p>That&#8217;s all for now. This development cycle is coming to a downslope, most of the big ideas for 0.28 have reached trunk, but of course there are always surprises in Santaeus&#8217; sleigh. So I don&#8217;t have horizon prognostication for you about what might appear next beyond the polish, balance tweaks, and bug fixing that make up the later end of the release cycle. Until next time, happy crawling!</p>
<p style="text-align: right;"><em>The energy of discovery flows out of the trunk update and into your fingertips!</em></p>


guid: http://crawl.develz.org/wordpress/?p=5156
title: Trunk Updates 17 January 2022 and Tournament Announcement
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-17-january-2022-and-tournament-announcement
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The trunk updates are haunted by guilt!</em></p>
<p>Hello crawlers, and welcome to the first trunk update of the new year! As hinted in the previous update, the 0.28 development cycle is nearing a close, and with that we are pleased to announce a <em>tentative tournament start date</em> of <strong>Friday, February 4, 2022, at 20:00 UTC</strong><em> </em>for the sixteen day 0.28 Release tournament. We are chilling trunk into a feature freeze and will be focusing on bug fixes and balance in the coming three weeks, and plan to release 0.28 before the start of the tournament. More details about the tournament, including a tournament website and rules changes, will be available here soon!</p>
<p>With that announced, let&#8217;s dive into the 215 commits that have arrived in trunk since the start of the year (more or less). These commits included community contributions from <em>Aliscans</em>, <em>DreamDust</em>, <em>Zhang Kai</em>, <em>perryprog</em>, <em>nlavsky</em>, <em>Goratrix</em>, and a new rare Temple layout featuring Ignis by <em>pdpol. </em>In total, in these commits were 72 bug fixes spread among GitHub issues, Mantis issues, reports directly to the devteam, and community contributions to fix a bug the fixer noticed. Along side the polish and gameplay balance, UI improvements are always something we&#8217;re happy to include, and team member advil has done extensive work to make arrow keys and menu help more consistently available across the UI.</p>
<p>The last &#8220;big ticket&#8221; block of changes to land was to the <strong>Abyss</strong>, let&#8217;s open with that suite of changes:</p>
<ul>
<li><strong>Random items</strong> no longer generate in the abyss.</li>
<li>Kills in the abyss <strong>grant no reward</strong> (experience and piety), though Uskayaw still rewards damage and Yred still grants zombies.</li>
<li><strong>Exploring</strong> no longer grants piety in the abyss.</li>
<li>The chance of <strong>plunging deeper</strong> into the abyss on teleport is increased.</li>
<li><strong>Wretched stars</strong> only apply a single temporary mutation per use of corrupting pulse, and use it less often.</li>
<li>The abyssal rune, staircases, and exits are <strong>more common</strong>.</li>
<li>On collecting the <strong>abyssal rune</strong>, the player gains two potions of experience worth of experience.</li>
</ul>
<p>With two-hundred commits there&#8217;s still a lot of churn, and although at this stage we&#8217;re much more focused on balance, there&#8217;s also a new goody or two. Let&#8217;s dive in!</p>
<ul>
<li><strong>New spell: Anguish (L4 Hex/Necromancy)</strong>. Gives monsters in sight that fail a Will check the anguished status. Whenever an anguished monster deals damage it also takes the same amount of damage.</li>
<li><strong>Shield penalties</strong> now work more like armour skill; the penalty scales down as skill is increased from 0 up to 27, based on a shield encumbrance rating and species factor. Skill 27 is required to completely remove the penalty, regardless of size.</li>
<li><strong>Spider</strong> is no longer quiet, and completely open level layouts are much rarer</li>
<li><strong>Ghost moths</strong> toned down in melee</li>
<li>orbs of <strong>mayhem</strong> are now disliekd by <strong>Cheibriados</strong>, and <strong>energy</strong> disliked by <strong>Trog</strong></li>
<li>the <strong>robe of Misfortune</strong> loses its unequip penalties</li>
<li>Ashenzari&#8217;s <strong>skill boost</strong> is turned down</li>
<li>Wendigo&#8217;s <strong>seracfall damage</strong> is reduced</li>
<li><strong>To-hit penalties</strong> from armour and shields are removed</li>
<li><strong>Aux attack chance</strong> and damage is now based on XL instead of attribute for all aux attacks.</li>
<li><strong>Offhand punch</strong> chance now starts at 5% chance and scales with unarmed combat skill to 50% at max skill.</li>
<li><strong>Dragon form</strong> gets 2 extra levels of fangs.</li>
<li>The UI for interacting with <strong>frenzied monsters</strong> is greatly improved.</li>
<li>The player gets XP for kills by and of <strong>frenzied monsters</strong>.</li>
<li><strong>Jiyva</strong> no longer grants piety for exploring the slime pits</li>
<li>The<strong> poison dart</strong> generation rate is dropped</li>
<li>Assorted tweaks to <strong>Spider</strong> monster band mixes and sizes</li>
</ul>
<p>Well, that&#8217;s all for now! If you&#8217;re interested in helping in the release effort, drop us a line in #crawl-dev on libera.chat, check out the open issues on GitHub, or just keep playing and discussing trunk as the balance and tuning come in.</p>
<p>On the horizon, well, I opened with it. There&#8217;s the 0.28 Release and Tournament! The team is going to be focusing on getting release-ready in the next 3 weeks. Until next time, happy crawling!</p>
<p style="text-align: right;"><em>The trunk update hits you! The trunk update is wracked by anguish!!! The trunk update dies!</em></p>


guid: http://crawl.develz.org/wordpress/?p=5165
title: 0.28 Tournament Page and Schedule
author: ebering
url: https://crawl.develz.org/wordpress/0-28-tournament-page-and-schedule
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Hello crawlers! We&#8217;ve finalized the schedule for the <strong>0.28 tournament</strong>, which will run from <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220204T20">Friday February 4th, 8pm UTC</a> to <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220220T20">Sunday February 20th, 8pm UTC</a>. Please bookmark the <strong><a href="https://crawl.develz.org/tournament/0.28/">official 0.28 tournament page</a> </strong>and use this page to track your progress throughout the tournament. We&#8217;ll continue to update this page with any rules changes. In coming days you may see summaries of test data in the &#8220;Overview&#8221; page taken from recent 0.28 beta games. Don&#8217;t worry, we&#8217;ll reset everything before the tournament starts.</p>
<p>We&#8217;re in the process of finalizing the 0.28 release which will appear in the next couple of days, and 0.28 is being rolled out to our online servers.</p>


guid: http://crawl.develz.org/wordpress/?p=5168
title: 0.28 “The Rise and Fall of Ignis Zotdust and the Spiders from Hell”
author: gammafunk
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
url:
https://crawl.develz.org/wordpress/0-28-%e2%80%9cthe-rise-and-fall-of-ignis-zotdust-and-the-spiders-from-hell%e2%80%9d

content:
<p>We are pleased to announce the release of Dungeon Crawl: Stone Soup 0.28 “The Rise and Fall of Ignis Zotdust and the Spiders from Hell”!</p>
<p>DCSS 0.28 features a new Cinder Acolyte zealot background that begins the game worshiping a new god, Ignis, the Dying Flame. Cinder Acolytes start with the new Scorch spell, a -1 flaming branded weapon of choice, and five stars of piety. Ignis worshipers cannot gain piety, losing it only when using Ignis&#8217; abilities. The abilities are Fiery Armour, which grants AC and retaliation flame damage, Foxfire Swarm, which summons many foxfires, and Rising Flame, a one-time ability sends the player one level above. Once all piety is spent, players will typically abandon for another god, and Ignis dies forever when wrath is over.</p>
<p>The Hell branches have been revised to offer substantially more challenge. Each branch features a distinct and persistent negative effects that applies while in the branch along with timed effects common to all Hells. The branches all have revised monster set with many new monsters and smaller layouts for the first six levels, each of which has only one downward staircase. The Spider branch has likewise been retooled with a roster of new creepy-crawlies, new and revised end maps, and updates to the existing layouts and vaults.</p>
<p>A new magical orb item type exists for the shield slot. Each orb type grants an ego with a powerful passive effect, such as a 3-radius halo or a reduction of monster willpower at the expense of your own. The god Yredelemnul has been completely redesigned, granting passive reaping and accepting the reaped zombies as payment for its dark abilities. Okawaru and Jiyva have also seen significant, though not quite as sweeping, changes.</p>
<p>A batch of new spells have been added, As always, there are dozens of new deadly vaults, and many more changes small and large alike!</p>
<p>Download DCSS 0.28 <a title="download" href="http://crawl.develz.org/download.htm">here</a> or <a title="play online" href="http://crawl.develz.org/play.htm">play it online</a> on one of many servers across the world! Packages for Windows and OS X are available now. Linux debs are coming soon. See the full list of changes on the release page <a href="https://github.com/crawl/crawl/releases/tag/0.28.0">here</a>.</p>
<p>The tournament starts on <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220204T20">Friday February 4th 8pm UTC</a> and runs through <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220220T20">Sunday February 20th 8pm UTC</a>, with all online 0.28 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.28/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220211T20">Friday February 11th 8pm UTC</a>.</p>
<p>Many thanks to all those who have <a title="credits" href="https://raw.githubusercontent.com/crawl/crawl/master/crawl-ref/CREDITS.txt">contributed</a> to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.28 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.28 and a warm welcome to the first-time contributors, listed in italics. (Let us know if someone is missing, this data is tricky to collect in bulk!)</p>
<blockquote><p>AdamPG, Alex Jurkiewicz, Aliscans, <em>ArmiesAndCastles</em>, <em>Brian Power</em>, Chris Landry, <em>Justin Clark Ong</em>, David Damerell, <em>DreamDust</em>, <em>Ed Dewey</em>, Elliott Bernstein, Gopall, Goratrix, Implojin, Jonathan<em> Klabunde Tomer</em>, <em>Kiëd Llaentenn</em>, MainiacJoe, Hurricos, <em>Matthew Daley</em>, <em>Monkooky</em>, Nikolai Lavsky, <em>Oneirical</em>, Perry Fraser, RojjaCebolla, <em>Santiago Acosta</em>, <em>Zachary Chandler</em>, Zhang<em> Kai</em>, amcnicky, dilly, floraline, John Stahara, Beargit, mgdelmonte, Nikolai Lavsky, Robert Gray, Paul Pollack, sdynet</p></blockquote>


guid: http://crawl.develz.org/wordpress/?p=5180
title: 0.28 Tournament Results
author: PleasingFungus
url: https://crawl.develz.org/wordpress/0-28-tournament-results
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The <a title="0.28 tournament" href="https://crawl.develz.org/tournament/0.28/overview.html">0.28 tournament</a> is over. On behalf of the Dev Team, many thanks to all the server admins, outside contributors, bug reporters, and to the many DCSS players who made the 0.28 release and tournament possible!</p>
<p>For 16 days, players could compete for tournament points and banners by playing 0.28 games on the public servers. This was the fourth tournament held after the major revision of the tournament rules done for 0.25. There were changes based on previous input, and once again we&#8217;ve collected the feedback posted on tavern and reddit.</p>
<p>Congratulations to <a href="https://crawl.develz.org/tournament/0.28/players/yermak.html">Yermak</a>, perennial champion, for once again taking first place in the <a href="https://crawl.develz.org/tournament/0.28/all-players-ranks.html">individual competition</a>, with a spectacular 111,412 points! Over the course of the tournament, Yermak won 43 games, including a streak of 13 games (second-best in tournament), an 76M high score run (first in tournament), 30 Nemelex&#8217; Choice combos, 4 species high scores, 3 background high scores, and 19 tier 3 banners.</p>
<p>Other big winners:</p>
<ul>
<li><a href="https://crawl.develz.org/tournament/0.28/players/acrobat2.html">Acrobat2</a>: 2nd place overall (101,150 points, 34 wins), longest streak (18 games), 4 species high scores &#038; 1 background high score, tied for lowest XL win at 16, all level 3 banners completed.</li>
<li><a href="https://crawl.develz.org/tournament/0.28/players/fagentul.html">Fagentul</a>: 3rd place overall (89,909 points, 21 wins), 5 species and background high scores, tied for lowest XL win at 16, 16 level 3 banners completed, 8-win streak.</li>
<li><a href="https://crawl.develz.org/tournament/0.28/players/p0werm0de.html">p0werm0de</a>: Fastest realtime win (an auspicious 27 minutes 44 seconds), 2nd place low turncount, 8th place overall.
<li><a href="https://crawl.develz.org/tournament/0.28/players/caminho.html">Caminho</a>: Lowest turncount win in a scanty 8,492 turns.
<li><a href="https://crawl.develz.org/tournament/0.28/players/sweetandcool.html">sweetandcool</a>: A third tie for lowest XL win at XL 16; 14th place overall.
<li><a href="https://crawl.develz.org/tournament/0.28/players/borgnjorsvilecrotch.html">Lici The Crawler</a> (alias Borgnjor&#8217;s Vile Crotch): First win of the tournament.
<li><a href="https://crawl.develz.org/tournament/0.28/players/ge0ff.html">Ge0ff</a>, <a href="https://crawl.develz.org/tournament/0.28/players/rubberychicken.html">RubberyChicken</a> and <a href="https://crawl.develz.org/tournament/0.28/players/noscrollmutbaddesign.html">NoScrollMutBadDesign</a> all tied for most ziggurats completed at 27 apiece.
</ul>
<p>The <a href="https://crawl.develz.org/tournament/0.28/teams.html">clan competition</a> was won by <a href="https://crawl.develz.org/tournament/0.28/clans/gozig-or-gozag-ge0ff.html">Gozig or Gozag</a> (captained by <a href="https://crawl.develz.org/tournament/0.28/players/ge0ff.html">Ge0ff</a>) with 105,656 points, sealing their victory by winning 40 Nemelex Choice combos to second place&#8217;s 30. In a close 2nd place was <a href="https://crawl.develz.org/tournament/0.28/clans/that-felt-unstrangely-rewarding-acrobat2.html">That felt unstrangely rewarding</a> (captained by <a href="https://crawl.develz.org/tournament/0.28/players/acrobat2.html">acrobat2</a>) with 98,434 points, and first place in the Clan Combo Highscores and Streak Length leaderboards. 3rd place was held by <a href="https://crawl.develz.org/tournament/0.28/clans/teamsplat-shummie.html">Team Splat</a> (captained by <a href="https://crawl.develz.org/tournament/0.28/players/shummie.html">shummie</a>) with 85,513, following up on last year&#8217;s 2nd place ranking by maxing out the Clan Low Turncount Win leaderboard.</p>
<p>Thank you to everyone for playing! I hope you all had fun.</p>
<p>To close out, here are some overall tournament statistics (with the 0.27 numbers in brackets for comparison). 2,559 players started a game at some point during the tournament; of those 2,504 completed a game without quitting or leaving the dungeon sadly orbless. This definition of “player” (players who completed a non-boring game) is used for subsequent calculations and was used for the 0.27 statistics.</p>
<ul>
<li>Players: 2,504 (0.27: 2,474)</li>
<li>Total player time: 28,008 hrs (0.27: 27,502 hrs)</li>
<li>Average player time: 11.2 hrs (0.27: 11.2 hrs)</li>
<li>Games played: 69,201 (0.27: 67,203)</li>
<li>Players who got a rune: 742 (0.27: 734)</li>
<li>357 winners and 1160 wins (0.27: 404 winners and 1571 wins)</li>
<li>Win rate: 1.68% (0.27: 2.34%)</li>
<li>Proportion of players using WebTiles: 96.6% (0.27: 94.4%)</li>
</ul>
<p>The 0.29 season has begun, and changes are already in motion. Watch this space for further information as the 0.29 trunk update posts begin to flow. Until next time, happy crawling!</p>


guid: http://crawl.develz.org/wordpress/?p=5194
title: Trunk updates 6 March 2022
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-6-march-2022
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The trunk updates gesture, and weapons take to the air!</em></p>
<p>Hello crawlers, and welcome to the first trunk update post of the 0.29 development cycle. The dust has settled from the 0.28 tournament, and a bug fix release for 0.28 will appear soon. Meanwhile, trunk development has taken off as various members of the team and the community get to work on new exciting projects. First off we have several new vaults by community contributors <em>DreamDust</em>, and <em>spicyCebolla</em>, as well as tweaks, fixes, and text from <em>Antern, chequers, CipHuk, Elliot Bernstein, Diego Essaya, Implojin, perryprog, </em>and <em>robertxgray</em>. Many thanks to the community contributors that keep crawl alive and healthy! Now let&#8217;s turn to the gameplay changes since 0.28.</p>
<h2>Launcher ammo is removed</h2>
<p>You read that correctly: player launchers now have unlimited ammo. Along with this are changes and tweaks to bring balance in this new, free firing world:</p>
<ul>
<li><strong>ranged weapons</strong> have had their stats adjusted downward</li>
<li>the<strong> Hunter</strong> background no longer starts with a short sword and no longer has a throwing option</li>
<li><strong>Kobolds</strong> now have +3 crossbows aptitude</li>
<li><strong>Armour penalties</strong> apply to ranged weapon attack delay</li>
<li><strong>Launcher damage</strong> scales with dex instead of str</li>
<li><strong>Spriggans</strong> can no longer wield triple crossbows, and fustibali are 2-handed.</li>
</ul>
<h2>Several new Uniques stalk the dungeon</h2>
<ul>
<li><strong>Parghit the Mighty. </strong>Following the example of Maggie/Margery, Parghit is an &#8220;alt-timeline&#8221; version of Pargi. Parghit appears in late Depths, Zot, Crypt and Tomb, and sports high-enchantment golden dragon scales (with a 1% chance of wearing the Dragon King) and an astonishing 27 HP regenerated per turn. He&#8217;s definitely killable, but best not underestimated.</li>
<li><strong>Josephina the Ice-Lich. </strong>As one more &#8220;alt-timeline&#8221; unique, Josephina is an upgraded version of Josephine. She appears in late Vaults, Depths, Crypt, Tomb, and Zot, and borrows Wendigo Seracfall (turning simulacrula into 3d33 bombs) to add to her classic Ghostly Fireball. She comes with a guaranteed randart ring of ice.</li>
<li><strong>Amaemon the Diabolical Poisoner.</strong> Crawl&#8217;s first demonspawn unique, appearing in mid-D, early Lair and Orc:1. Alongside his orange demon companion, Amaemon summons scorpions (the classic returns!) and casts Concentrate Venom, upgrading his allies&#8217; poison into curare. To encourage players to try to kill him, he wields a demon whip of venom, with 1% chance of the unique Snakebite.</li>
<li><strong>Lodul the Thundrous. </strong>Lodul is an ironbound thunderhulk who&#8217;s been sent to patrol outside the Vaults, in late D. He&#8217;s got a weaker version of Call Down Lightning and a big band of ogres and two-headed ogres to help him keep distance from the player. Defensively, he looks a lot like Erolcha.</li>
<li><strong>Vv the Exile. </strong>Vv is a combination of ironbound frosthearts (Creeping Frost) and Salamander Tyrants (Eruption/Pyroclastic Surge), appearing in Depths, Zot, Crypt, and Tomb. She&#8217;s themed as a new species, the Stoneborn, sentient earth elementals. She also has strong defenses and both flaming and cold melee.</li>
<li><strong>Zenata, Seeker of Sheza. </strong>Zenata is another late-game unique (appearing in Depths, Zot &amp;c), borrowing a bit from the Desolation of Salt and putting a twist on it. She&#8217;s taken the ragged hierophants&#8217; &#8220;Resonance Strike&#8221;, which is like airstrike that gets bonus damage from adjacent constructs instead of from open spaces, and uses it in conjuction with a new &#8220;Sheza&#8217;s Dance&#8221; spell, which summons 2-4 dancing weapons. She comes with a high-tier weapon with some plusses already on it.</li>
</ul>
<h2>Monster energy and attacks of opportunity</h2>
<p>A major mechanical change has landed in trunk. &#8220;Random energy&#8221; was introduced long ago in Stone Soup&#8217;s history as an attempt at deterring tedious-but-optimal levels of pillar dancing and luring, while permitting shorter trips. It has not been successful in that role, and additionally offered a &#8220;get out of jail free-ish&#8221; card in many fights against certain monsters. The spirit that motivated it also motivated previous devteam attempts to meet the problem with a better solution, such as the dynamic-monsters experimental during the 0.17 cycle. This time around we&#8217;re tinkering with the mechanic again, in the following form:</p>
<ul>
<li>Monster move energy is no longer randomized.</li>
<li>Monsters adjacent to the player gain a 1/3 chance to gain a free* attack if the player moves away from them. Wu Jian martial attacks &amp; rampaging give immunity, and monsters that don&#8217;t move fast enough to follow you can&#8217;t attack. *Terms and conditions apply: the monster must have sufficient energy to be able to move the next turn (monsters slower than the player don&#8217;t get this chance on the turn they fall behind), and monsters that attack slowly don&#8217;t get a full refund on the energy used for the attack, only the amount of a normal turn.</li>
<li>Ogres, 2-headed Ogres, Ogre magi, and Thunderhulks attack more slowly in melee.</li>
<li>Early D spawns are nerfed a bit, with top killers less common and Harold moved later.</li>
</ul>
<p>In addition to the direct tuning changes above, backgrounds have their starting kits improved a bit with extra consumables, to increase tactical options in the very early game. See the next section.</p>
<h2>Expanded starting kits</h2>
<p>Many backgrounds get improved starting kits with a new consumable or two to help with the very earliest moments of the game when tactical options are limited.</p>
<ul>
<li>All <strong>&#8216;mages&#8217; (&#8216;pure casters&#8217;)</strong> start with a potion of magic.</li>
<li><strong>Gladiators</strong> get a throwing weapon (one); a javelin, large rock, or boomerang.</li>
<li><strong>Monks</strong> get a potion of ambrosia.</li>
<li><strong>Hunters</strong> get a scroll of immolation.</li>
<li><strong>Brigands</strong> get an additional poisoned &amp; curare dart.</li>
<li><strong>Artificers</strong> trade their chesspiece for nine charges of iceblast.</li>
<li><strong>Wanderers</strong> get an additional random potion or scroll.</li>
<li><strong>Abyssal Knights</strong> start at 60 piety (just over 2*) instead of 38 (just over 1*). This should allow them to use Bend Space if needed.</li>
<li><strong>Chaos Knights</strong> get Artificer&#8217;s xom chesspiece.</li>
<li><strong>Transmuters</strong> get a potion of lignification.</li>
<li><strong>Warpers</strong> get more boomerangs of dispersal.</li>
<li><strong>Arkane Markspersons</strong> get a scroll of vulnerability.</li>
<li><strong>Enchanters</strong> get another potion of invisibility.</li>
</ul>
<h2>Everything else</h2>
<p>Along with big-ticket changes there&#8217;s the usual froth and fizz of trunk development, with little tweaks and small features all over.</p>
<ul>
<li>Unrand tweaks:</li>
<ul>
<li><strong>the Glaive of the Guard</strong> is reduced to +5</li>
<li><strong>the arbalest &#8220;Dammnation&#8221;</strong> is reduced to firing 2d14 explosions</li>
<li><strong>Gyre &amp; Gimble</strong> are now +12</li>
<li><strong>the Woodcutter&#8217;s axe</strong> is now a battleaxe</li>
</ul>
<li><strong>Long and short blade</strong> damage scales with <strong>Dexterity</strong> instead of Strength.</li>
<li><strong>Merfolk&#8217;s polearms</strong> aptitude is reduced to +3, with long blades increased to +2</li>
<li><strong>Mummy necromancy enhancers</strong> now appear at XL 3 and 13</li>
<li>Most <strong>zombies</strong> and <strong>skeletons</strong> are removed from Dungeon generation.</li>
</ul>
<p>That&#8217;s all for now! The team is still getting organized around 0.29 priorities, so through the haze of Gehenna it&#8217;s hard to make out what&#8217;s coming. One big project is an overhaul of Necromancy, particularly the corpse exploitation spells; it&#8217;s still in the early phases so keep an eye out to see what sort of dark abomination comes out of the work. Until next time, happy crawling!</p>
<p style="text-align: right;"><em>The trunk updates hit you as you move away! Ouch! That really hurt!!! You die&#8230;</em></p>
<p>&nbsp;</p>
<p>&nbsp;</p>


guid: http://crawl.develz.org/wordpress/?p=5200
title: Trunk Updates 19 June 2022
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-19-june-2022
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>Kikubaaqudgha calls the trunk updates from the earth!</em></p>
<p>Hello crawlers, and welcome back to Trunk Updates. While I&#8217;ve been away in a far corner of Pandemonium, my fellow devils have slacked off and left this space blank! Well, not slacked exactly, because they&#8217;ve been hard at work in Tar:7, as the length of this post portends. There have been several big moves, in the direction of <strong>item sets</strong>, <strong>a necromancy overhaul</strong>, and <strong>a new species</strong>, as well as balance adjustments following the opportunity attacks change and of course a wild grab bag of other changes. Before we dive into those, let&#8217;s mention community contributions since we last wrote: <em>dilly </em>sent in several new guarded unrand vaults, <em>sdynet </em>tweaked the rings of the Octopus King, and <em>Aliscans, chequers, nlavsky, perryprog, </em>and <em>spicyCebolla </em>each sent in various fixes and adjustments. I know there&#8217;s a big backlog of pull requests on the GitHub right now, but with luck we&#8217;ll bring in a bunch more new community work in the next month. Until then, let&#8217;s look at the rest of the changes!</p>
<h2>Item sets</h2>
<p>In an effort to increase run-to-run variation, experiment with new interesting item effects, while not increasing item pressure, the new notion of an item set is introduced. Only one type of item from a set can generate normally in a given game (though certain special situations, like rare vaults or shops, or perhaps a forgotten god, might bend this rule). Currently there are two item sets:</p>
<ul>
<li>The <strong>hex wand set</strong>: alternating between Paralysis and Charming.</li>
<li>The <strong>bolt wand set</strong>: alternating between Acid, and two new wands: Light and quicksilver.</li>
<ul>
<li><strong>wands of light</strong> fire a bolt of blinding light, checking AC and EV but no resists. The beam blinds what it hits, and deals  damage identical to an acid wand.</li>
<li><strong>wands of quicksilver </strong>fires a quicksilver bolt, dispelling the magical effects of targets it hits. Does slightly more damage than acid.</li>
</ul>
</ul>
<p>Watch this space for future exciting item sets!</p>
<h2>Corpseless necromancy</h2>
<p>As foreshadowed in the last post, hellmonk and PleasingFungus have been hard at work on overhauling Necromancy to not use corpses directly, as well as make Necromancy less awkward to use and give Kikubaaqudagh some bone polish. Their efforts have paid off, and crawl&#8217;s Necromantic magic now deals directly in the shadow between life and death, instead of meddling with the long-dead.</p>
<ul>
<li><strong>Kikubaaqudgha</strong> is buffed:</li>
<ul>
<li>The <strong>pain brand</strong> can be applied to ranged weapons.</li>
<li><strong>Torment</strong> no longer requires a corpse sacrifice.</li>
<li><strong>Corpse Delivery</strong> is removed and replaced with <strong>Unearth Wretches</strong>, which summons some paralysed, 1 HP, 0 will monsters that die in a few turns, to use Necromancy spells on.</li>
</ul>
<li><strong>Animate Dead</strong> now gives a duration, and for the duration killed monsters with corpses have a power-dependent chance to become zombies. The zombies are permanent as long as the player is on the level, and are destroyed by a subsequent cast of Animate Dead.</li>
<li><strong>Simulacrum</strong> is now a smite-targeted spell that gives an irresistable status. Monsters that die with this status spawn a power-dependent number of simulacra.</li>
<li><strong>Death Channel</strong> cannot be re-cast while active.</li>
<li><strong>Corpse Rot</strong> is now Level 4 Necromancy/Poison/Air. When cast it produces miasma for each corpse-producing monster killed in sight, scaling with power.</li>
<li><strong>New Spell:</strong> Necrotize (L1 Necromancy). Necrotize deals damage identically to the old Pain spell, but additionally gives the monster a short-lived status that causes it to animate as a skeleton on death (if it has a skeleton). It replaces Animate Skeleton and Pain in all player usable places.</li>
<li>The spells Animate Skeleton, Pain, and Excruciating Wounds are removed.</li>
</ul>
<h2>Meteorae</h2>
<p>The roiling success of hellmonk&#8217;s April fools Mayflytaurs&#8217;s mechanic comes back in a modified form. Welcome <strong>Meteorae</strong>!</p>
<p>Meteorae have an exciting variety of bonuses &#8211; great attributes and aptitudes across the board, passive mapping, and exploration healing, regaining HP and MP when viewing new territory. In exchange, they have one big downside: instead of getting 6,000 turns of Zot clock for each floor, they get 600! Meteorae are humanoid beings. (In the night sky, they look like dots because they&#8217;re very far up.) Hat tip to Neil Gaiman&#8217;s Stardust. They also glow in the dark, with permanent backlit status (not halo!), that they can&#8217;t escape by transmuting (Dithmenos can hide them, though!).</p>
<p>Along with their introduction comes a softening of the <strong>Zot clock</strong>, a slight move away from &#8220;instant death&#8221;. The Zot clock now applies very heavy draining the first time it trips, and gives the player an additional set of turns. Only at very low max HP will it outright kill the player.</p>
<h2>Balance adjustments</h2>
<p>The introduction of attacks of opportunity was a significant shift, and there have been several follow up changes to adjust balance while keeping the tighter tactics they introduce.</p>
<ul>
<li><strong>Some early game monsters</strong> are nerfed a bit in terms of base damage.</li>
<li><strong>Consumable generation</strong> is increased.</li>
<li>Monsters slower than the player don&#8217;t get <strong>opportunity attacks</strong>.</li>
<li>The <strong>maximum duration of paralysis</strong> a monster can inflict is reduced to 5 turns.</li>
<li><strong>Dungeon shafts</strong> attempt to send the player to a location outside of LOS of monsters.</li>
<li>Using all <strong>un-explored stairs and hatches</strong> is now fast.</li>
</ul>
<h2>The rest</h2>
<p>Of course, this wouldn&#8217;t be a trunk updates post without a potpourri of other assorted changes. First of all, advil has done a host of behind the scenes work, improving the shopping menu, the plumbing of various bits of UI and save handling. And of course, Sastreii&#8217;s artistic endeavors continue to beautify the dungeon as well as clarify UI elements: you might have noticed the flashy new status icons for various status effects that are applied to monsters. Read on for the rest:</p>
<ul>
<li><strong>Timed hell effects</strong> no longer apply on a timer, and instead have a chance to apply on each level entry.</li>
<li><strong>Hunter</strong>&#8216;s starting dexterity is increased.</li>
<li><strong>Positive stat mutations</strong> now give +4 to the primary stat and -1 to the other two.</li>
<li><strong>Formicids&#8217; base dexterity</strong> is increased by 3.</li>
<li><strong>Formicids</strong>  can now wield giant (spiked) clubs.</li>
<li><strong>Demonspawn</strong> demonic touch damage is doubled.</li>
<li><strong>Uskayaw</strong> grants piety for ally damage.</li>
<li>Lugonu&#8217;s <strong>Bend Space</strong> is adjusted. It no longer does damage, and makes a better effort to blink monsters away from the destination.</li>
<li><strong>Stones</strong> always mulch when thrown.</li>
<li><strong>Shadow dragon scales</strong> AC increased</li>
<li>Weapon descriptions now display a &#8220;<strong>damage rating</strong>&#8220;, describing the contributions to weapon damage from various sources.</li>
<li><strong>New item: scroll of poison</strong>. When read the scroll of poison fills every unoccupied space on the screen with poison clouds for 10-20 turns.</li>
<li><strong>+MP items</strong> remove MP when they are taken off.</li>
<li><strong>New unrand:</strong> the +8 trishula &#8220;Condemnation&#8221; {holy, Anguish Fly rN+}. Inflicts 4-8 turns of the Anguish status on targets it hits.</li>
<li><strong>New unrand:</strong> the +3 mountain boots {rDislodge, rC+}. Prevent the player from being involuntarily moved by physical means (trampling, shafts, tentacles, etc.).</li>
<li>The <strong>amulet of Vitality</strong> now gives two pips each of Regen and RegenMP, and is now evil (Borgnjor is said to have made or used it or been involved in it somehow).</li>
<li>Scrolls of <strong>holy word </strong>are removed</li>
<li><strong>Animated Armour</strong> now flies.</li>
<li><strong>Sandblast</strong> no longer uses stones as ammo. Instead it takes twice as long to cast as a normal spell, and has its range increased by one.</li>
<li><strong>Airstrike&#8217;s</strong> bonus from empty space is doubled.</li>
<li>Finally, <a href="https://github.com/crawl/crawl/commit/05f53baf05f6">Deep Dwarves are removed</a>.</li>
</ul>
<p>Wow! It&#8217;s been a few months since we last wrote and a lot has happened. Still more is on the horizon, though through the haze what exactly remains as mercurial as a quicksilver elemental. So, until next time, happy crawling!</p>
<p style="text-align: right;"><em>Whisper &#8220;Farewell&#8221; when you leave, Dimli&#8230;</em></p>


guid: http://crawl.develz.org/wordpress/?p=5210
title: Trunk Updates 18 July 2022
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-18-july-2022
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>The trunk updates twirl on the spot.</em></p>
<p>Hello crawlers, and welcome to the latest trunk updates! It&#8217;s a hot summer here in Gehenna, where the devil&#8217;s team is toiling away. Before we dive into the latest diabolical ways to die, mark your calendars. The tentative dates for the <em>0.29 release tournament</em> are <strong>26 August 2022 to 11 September</strong>. Watch this space for more information as the 0.29 development cycle winds down and we ready the new version for release.</p>
<p>We&#8217;ve got a big trunk update for you. Since last time we&#8217;ve added community contributions from nlavsky, robertxgray, Goratrix, Zhang Joke, Cebolla Sunbeam, chequers, perryprog, amcnicky, nikheizen, Brian Power, Aliscans, David Damerell as well as welcomed first-time contributors <em>4Hooves2Appendages</em>, <em>catern</em>, <em>Sentei</em>, <em>Reono</em>, <em>ruxi</em>, and <em>Rosstin Murphy</em>. Some of these are bug fixes, grammar tweaks, quotations and dialog, and vaults, the little stuff that all together gives crawl its polish, but there are also some bigger-ticket items in the round up. If you&#8217;re interested in helping out with the effort to make crawl, <a href="https://github.com/crawl/crawl/#how-you-can-help">click here for more information</a>!</p>
<p>With that said, let&#8217;s dive into what&#8217;s been shaking in trunk.</p>
<p>First up, a new monster: the <strong>starflower</strong>, contributed by <em>Cebolla Sunbeam</em>. A stationary plant/tentacle hybrid monster with harpoon shot, reach 3 distortion melee, reach 2 melee, and an additional distortion melee attack. Meant as a vault-placed monster for abyssal vaults, guarding an exit or a rune or what have you, as a threat that&#8217;s dangerous to be close to, but can be neutralized if the harpoon shot can be blocked or avoided somehow.</p>
<p><em>Sentei</em> <strong>adjusted the -Scroll and -Potion mutations</strong>. They are now 2-tier mutations in the regular bad mutation pool. The Ru sacrifice corresponds to the 2nd tier. Follwing up on Senti&#8217;s work, <strong>teleportits and berserkitis</strong> are also now two tiers.</p>
<p><em>nikheizen</em> sent us a design to reflavour and re-mechanic <strong>gnoll shamen</strong>, turning them into <strong>gnoll bouda</strong>. They have a new spell set: Weakening Gaze, Hunting Call (like battlecry but gives swiftness instead of might), and heal other. The name bouda is a nod to the bouda of Northeast African folklore: shapeshifters that turn into jackals. They have the power of the Evil Eye which curses others with spectacular misfortune.</p>
<p>Temples are adjusted to place six to twenty-one altars, with a distribution that lightly favors the center of that range. On a one percent chance, a rare temple is selected instead. Rare temples are to Lugonu, Jiyva, or Ignis, or have a particularly extreme number of altars. Designing a Temple map is a great way to start contributing: there&#8217;s a particular need for Temples that support 7-8, 11, 17, 19, or 20 altars. Read more about map making for crawl <a href="https://github.com/crawl/crawl/#map-making">here</a>!</p>
<p>Support for item reskins has been added. Crazy Yiuf&#8217;s has his <strong>hammers once more</strong>. <strong>Sigmund</strong> and <strong>Reapers&#8217;</strong> scythes are now reskinned halberds (so you can actually use them if you want, instead of trap items), and all scythes except unrand scythes are removed.</p>
<p>The<strong> Bows, Crossbows, and Slings</strong> skills are combined into a single <strong>Ranged Combat</strong> skill. Fustibali are removed. Hunting slings are renamed to slings. Launcher stats and spawn rates and locations are adjusted to give the ranged skill greater strategic depth.</p>
<p>Continuing to build on the item rotation concept, there is a <strong>new item: wand of roots</strong>, appears in item rotation with iceblast. Roots fires a 3&#215;3 fireball-sized effect which constricts its victims (like BVC), with a duration scaling with wand power.</p>
<p>And a grab bag of slightly smaller changes:</p>
<ul>
<li>a new <strong>silent ossuary map</strong></li>
<li><strong>Jiyva altars</strong> now place 2/3rds of the time at the entrance to Slime</li>
<li><strong>Hex miscasts</strong> now only slow the player, instead of also cancelling effects</li>
<li><strong>Lair</strong> now 5 floors</li>
<li><strong>Ijyb&#8217;s </strong>wand power is nerfed.</li>
</ul>
<p>That&#8217;s all for now. Expect the changes to slow down as we arrive into feature freeze in August and get ready for the release. Until next time, happy crawling!</p>
<p style="text-align: right;"><em>Which is more difficult, to awaken one who sleeps or to awaken one who, awake, dreams that he is awake?</em></p>


guid: http://crawl.develz.org/wordpress/?p=5218
title: Trunk Updates 14 August 2022 and Tournament Announcement
author: ebering
url: https://crawl.develz.org/wordpress/trunk-updates-14-august-2022-and-tournament-announcement
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p><em>You hear the tinkle of a tiny bell. The trunk updates and a butterfly come into view!</em></p>
<p>Hello crawlers! Welcome to the last trunk update of the 0.29 cycle. We&#8217;ll be branching off the 0.29 beta branch and rolling it out to the servers soon, in anticipation of an upcoming release and tournament starting<strong> Friday, August 26, 2022, at 8:00pm UTC</strong>. Watch this space for the tournament website soon!</p>
<p>As we&#8217;ve been in feature freeze for part of the time since the last update post there&#8217;s not much to report, but here&#8217;s the latest on what will round out the version. Expect to see a few more new vaults and tweaks before the beta, and don&#8217;t hesitate to report any bugs you&#8217;ve noticed. Community contributor nlavsky has continued the stream of bug fixes; riverfiend did a deep dive in god wraths and did a lot of sanity checks, big thanks to them both! Here are the rest of the changes:</p>
<ul>
<li>A new <strong>scroll of butterflies</strong> summons some insectoid cover, alternating in a set with the scroll of fog</li>
<li>Sastrei has been hard at work drawing a slew of new tiles</li>
<li>Some <strong>spell balance</strong> tweaks:</li>
<ul>
<li><strong>Irradiate</strong> now applies more contamination</li>
<li><strong>Flame wave </strong>and <strong>sticky flame</strong> do more damage</li>
<li><strong>Sandblast</strong> is more accurate</li>
<li><strong>Frozen Ramparts</strong> and <strong>Ozocubu&#8217;s Refrigeration </strong>now check AC.</li>
<li><strong>Ozocubu&#8217;s Refigeration</strong> no longer applies -Potion, does more damage, but significantly reduced damage to monsters adjacent to each other (they huddle for warmth, cute!).</li>
</ul>
<li><strong>Arcane Markspersons</strong> starting sling is now +1, and they trade the scroll of vulnerability for a scroll of poison.</li>
<li>The <strong>roll attack</strong> of Palentonga is now quieter, and they gain +2 strength.</li>
<li>A lot of behind-the-scenes work in key code handling and UI menus has been done by <strong>a</strong><strong>dvil</strong>; the work has payed dividends in making new menu shortcut features easy to implement, visible in the use an item menu (toggle between wear/takeoff with !). Look forward to more of this in the next version.</li>
</ul>
<p>Along with these gameplay changes have been a host of bug fixes as we get the next version ready for launch. Get amped for the tournament, and until next time, happy crawling!</p>
<p style="text-align: right;"><em>The butterfly can only feel hate for you!</em></p>


guid: http://crawl.develz.org/wordpress/?p=5222
title: 0.29 “Shooting Stars”
author: PleasingFungus
url: https://crawl.develz.org/wordpress/0-29-shooting-stars
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We are pleased to announce the release of Dungeon Crawl: Stone Soup 0.29 “Shooting Stars”!</p>
<p>DCSS 0.29 features a sweeping rework of Necromancy, streamlining and upgrading many of its spells, along with similar improvements to the necromantic god Kikubaaqudgha. Ranged weapons have likewise been streamlined and reworked to be more satisfying to use.</p>
<p>For those seeking a special challenge, we&#8217;ve also added a new Meteoran species that heals when seeing new terrain and has great skills and attributes. However, Meteorans have to move fast, with only one-tenth the normal time per floor before they suffer heavy draining.</p>
<p>Many backgrounds now start with extra scrolls, potions, or other consumables, allowing more tactical options when faced with a sticky situation. Speaking of sticky situations, characters cast into the Abyss will find that monsters there once again give experience and piety, and an ultra-dangerous &#8216;deep abyss&#8217; is now accessible&#8230;</p>
<p>Many new item types have been added, like scrolls of poison and wands of quicksilver. Some of these &#8216;alternate&#8217;, with only one type of item from a set being generated in any given game. We&#8217;ve also added many exciting new unique monsters, from Amaemon the Diabolical Poisoner to the alt-timeline troll Parghit the Mighty.</p>
<p>Launchers, long blades and short blades all now use dexterity instead of strength when calculating damage, allowing new evasive playstyles. Monsters no longer gain and lose &#8216;random energy&#8217;, and can instead launch opportunistic attacks on fleeing players. As always, there are many more changes small and large alike!</p>
<p>Download DCSS 0.29 <a title="download" href="http://crawl.develz.org/download.htm">here</a> or <a title="play online" href="http://crawl.develz.org/play.htm">play it online</a> on one of many servers across the world! Packages for Windows and OS X are available now. Linux debs are coming soon. See the full list of changes on the release page <a href="https://github.com/crawl/crawl/releases/tag/0.29.0">here</a>.</p>
<p>The tournament starts on <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220826T20">Friday August 26th 8pm UTC</a> and runs through <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220911T20">Sunday September 11th 8pm UTC</a>, with all online 0.29 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.29/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220902T20">Friday September 2nd 8pm UTC</a>.</p>
<p>Many thanks to all those who have <a title="credits" href="https://raw.githubusercontent.com/crawl/crawl/master/crawl-ref/CREDITS.txt">contributed</a> to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.29 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.29 and a warm welcome to the first-time contributors, listed in italics. (Let us know if someone is missing, this data is tricky to collect in bulk!)</p>
<blockquote><p>4Hooves2Appendages, Aliscans, <em>Ernest Antanishin</em>, <em>Spencer Baugh</em>, Elliott Bernstein, David Damerell, DreamDust, dilly, Diego Essaya, Perry Fraser, Goratrix, Implojin, Alex Jurkiewicz,  Nikolai Lavsky, <em>neo-bop</em>, <em>Oleksiy Pavlikovsky</em>, <em>Rytris Petronis</em>, Brian Power, <em>Reono</em>, riverfiend, robertxgray, <em>Rosstin Murphy</em>, <em>ruxi</em>, sdynet, Jeffery Stager, Cebolla Sunbeam</p></blockquote>


guid: http://crawl.develz.org/wordpress/?p=5234
title: Trunk Updates 11 Jan 2023
author: PleasingFungus
url: https://crawl.develz.org/wordpress/trunk-updates-11-jan-2023
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Happy new year, crawlers! It&#8217;s been a little longer than usual since the last updates post, so this post will be a little longer than usual, too. To get you started, here&#8217;s a cool new splash screen from PhilosopherOpposite:</p>
<p><img src="https://github.com/crawl/crawl/raw/master/crawl-ref/source/dat/tiles/title_philosopheropposite_palentonga_paladin.png" alt="An Armataur Paladin, by PhilosopherOpposite." /></p>
<p>And off we go!</p>
<ul>
<li>New background: <strong>Reaver</strong>.
<ul>
<li>Reavers are warrior-mages, primarily focused on melee combat but with<br />
      situational Conjurations for tough situations.</li>
<li>New starting spell: <strong>Kiss of Death</strong> (L1 Conj/Necro). Does high draining damage to an adjacent enemy, at the cost of significant max HP drain to the caster.</li>
<li>New starting spell: <strong>Momentum Strike</strong> (L2 Conj/Tloc). Redirects the caster&#8217;s momentum for a damaging attack that doesn&#8217;t require line of fire, but locks the caster in place for some turns after a hit. (It cannot be recast until this effect expires.)</li>
<li>Reavers also start with <strong>Hailstorm</strong>, which is now L3 (formerly L4) and can now miss. Ice Elementalists lose Hailstorm, but get <strong>Summon Ice Beast</strong> in exchange.</li>
</ul>
</li>
<li>Species rework: <strong>Palentongas</strong> have become <strong>Armataurs</strong>.
<ul>
<li>Like Palentongas, Armataurs are large, scaled mammals, wearing bardings<br />
      instead of boots.</li>
<li>New species innate: <strong>Long Tongue</strong>, which <strong>doubles</strong> the effects of drinking potions, but makes reading scrolls take 50% longer.</li>
<li>New species innate: <strong>Rolling</strong> (innate Rampage &#8211; extra movement when moving toward enemies).
<li>Rolling Charge is now a spell, <strong>Vhi&#8217;s Electric Charge</strong> (L3 Tloc/Air). It sends the caster charging toward a nearby foe to deliver an electrified strike, phasing through targets in the way. <strong>Warpers</strong> now start with Vhi&#8217;s Electric Charge instead of Gell&#8217;s Gravitas.</li>
<li>Armataurs lose the Rolling Charge and Curl abilities.</li>
<li>Armataurs&#8217; aptitude have been adjusted across the board.</li>
<li>See <a href="https://github.com/crawl/crawl/commit/eb9d758594607a294226f720076e023a7eff41ca">these</a> <a href="https://github.com/crawl/crawl/commit/ba81c3259c708816d76a91b623ddd8d86483cbd9"> commits</a> for more explanation of the reasoning behind the changes.</li>
</ul>
</li>
<li>God rework: <strong>Lugonu</strong> has been substantially revised, with the goal of being a much more tempting pick for players unlucky enough to have been banished to the Abyss.
<ul>
<li>The Banishment ability is now available at ** piety (was ***). It fails less often, and when it is resisted, the victim will be malmutated instead, making them more vulnerable in future. (This does not apply to other sources of banishment, such as the monster spell or weapons of distortion.)</li>
<li>When another god&#8217;s wrath descends upon a worshipper, Lugonu now scornfully responds by banishing several nearby foes.</li>
<li>Altars to Lugonu are more than three times as common in the Abyss.</li>
<li>Converting to Lugonu in the Abyss now gives even more bonus piety.</li>
<li>The Abyssal Knight background and the Bend Space ability have been removed. (For more explanation of why, see <a href="https://github.com/crawl/crawl/commit/e63b5e06755426de847c50467f48fc9086cc1eea">these</a> <a href="https://github.com/crawl/crawl/commit/793202b3189c0a71dd71d3dc1be6e40c5faa5d07"> commits</a>.)</li>
</ul>
</li>
<li>New brand: <strong>Heavy</strong>.
<ul>
<li>Heavy weapons have +80% base damage and +50% attack delay. (This has proven to be pretty strong, so don&#8217;t be surprised if this gets toned down slightly before release.)</li>
<li>Both melee and ranged weapons can be Heavy.</li>
<li>The Vorpal brand has been removed. It simply wasn&#8217;t very exciting. Flaming and Freezing remain as very simple damage multiplier brands.</li>
<li>The Dark Maul and Sniper are both now Heavy instead of Vorpal, and have had various stat adjustments. (Sniper is now +27.) Other formerly Vorpal unrands have changed or lost brands and gained enchantment as needed.</li>
</ul>
</li>
<li>New unrands:
<ul>
<li>The <strong>Hermit&#8217;s Pendant</strong>, an amulet of faith that sets the wearer&#8217;s Invocations skill to 14 and their Evocations skill to 0. (Only for as long as you continue wearing the amulet, of course.)</li>
<li>The <strong>Force Lance</strong>, a Heavy +5 lance (a spear with +2 base damage, like Wyrmbane). When it lands a damaging hit, there&#8217;s a 1/3 chance of the target being knocked back, suffering extra damage if it collides with anything.</li>
<li>The <strong>Slick Slippers</strong>, which make all damaging melee attacks trample their wearer backward. They also provide immunity to constriction, engulfing, webbing and netting!</li>
</ul>
</li>
<li>New spells:
<ul>
<li><strong>Cigotuvi&#8217;s Dreadful Rot</strong> (L2 Necro/Air/Pois). Rots away the caster&#8217;s flesh to create a cloud of deadly miasma on their tile. Great for escapes, but also very fun with effects like Lesser Beckoning and potions of attraction! Replaces Conjure Flame in the Hedge Wizard starting spell set.</li>
<li><strong>Kindle Blastsparks</strong> (L3 Fire/Air/Tloc). Creates a cloud of &#8216;blastsparks&#8217; on the caster&#8217;s tile, which explode when any creature moves into them or when they are hit by fire damage. Usable as a land mine, as a way to create distance from enemies, or for other fun tricks. Replaces Conjure Flame in the Fire Elementalist starting spell set.</li>
<li><strong>Arcjolt</strong> (L5 Conj/Air). Electrocutes everything adjacent to the caster, everything adjacent to those creatures, etc, etc, in a chain.</li>
<li>(And of course Reavers&#8217; Kiss of Death and Momentum Strike are both new, too &#8211; see above!)</li>
</ul>
</li>
<li>Further item-related changes:
<ul>
<li>The <strong>Mad Mage&#8217;s Maulers</strong> have been reworked. They lose Int-3 and RegenMP and gain -Cast and MPVamp, causing the wearer to gain MP when they strike foes in melee. They now add as much damage when spending MP in melee, matching the normal infusion ego&#8217;s ratio of MP to bonus damage.</li>
<li>Rings of evasion are now +5 instead of +4, to be closer in value to other rings (eg AC/Dexterity). The Ring of the Hare, accordingly, goes from +8 to +10.</li>
<li>Majin-Bo is now a one-handed staff instead of a two-handed quarterstaff. (The HP vamp and not being able to wield an enhancer staff is plenty of drawback.)</li>
<li>Scrolls of butterflies now create dramatically more butterflies. They now alternate with scrolls of summoning instead of scrolls of fog, since the latter have a more unique and irreplacable effect. Watch this space!</li>
<li>All ranged weapons can now generate with electrocution, antimagic, and draining brands. Venom ranged weapons no longer appear.</li>
<li>Short sword attack delay reduced.</li>
<li>Evocable invisibilty and blink no longer have an MP cost.</li>
<li>Spells have been reshuffled between books, and a new Maxwell&#8217;s Memoranda book has been added.</li>
<li>Hand crossbows are very slightly weaker and slower.</li>
<li>The Lochaber Axe now causes temporary HP drain on unequip.</li>
</ul>
</li>
<li>Further character-related changes:
<ul>
<li>Arcane Marksmen (Markspersons?) have been renamed to Hexslingers, for stylistic and pun reasons.</li>
<li>Base regeneration increased by 0.2 HP/turn, to make resting faster in the early game.</li>
<li>Mummies have innate Faith, giving extra piety gain.</li>
<li>Reaching the end of the Zot clock now causes a smaller amount of permanent max HP loss (similar to Borgnjor&#8217;s Revivication), rather than temp HP drain. The Zot clock can no longer directly kill the player. (This change is intended to simplify the mechanic and to move away from a weird/nonstandard use of HP draining.)</li>
<li>Large species move at normal speed through shallow water.</li>
<li>rF- and rC- are added to the &#8216;bad mutations&#8217; pool.</li>
<li>Will is now capped at +++++. (When affected by the Will/2 status, the cap is applied before halving.)</li>
<li>Pale draconians&#8217; breath is no longer usable twice as often as others&#8217;.</li>
<li>Formicids can now shaft themselves in the Tomb of the Ancients.</li>
</ul>
</li>
<li>Further god-related changes:
<ul>
<li>When worshipping at a faded altar, the player is given a preview of three gods the altar might belong to. (Then you can back out if you don&#8217;t like the options, without suffering wrath.) Care to take a gamble on that altar to Sif Muna, Vehumet, or Trog?</li>
<li>Gozag&#8217;s corpse-to-gold effect now blocks enemies from creating simulacra.</li>
<li>Fedhas protects trees from being set on fire by the player.</li>
<li>Okawaru&#8217;s Heroism and Finesse abilities cost more piety.</li>
<li>Removed Ashenzari&#8217;s Astral Vision (automatically seeing through walls). It was very annoying for autotravel/resting/etc, had a surprising number of bugs, and Ashenzari was quite strong enough without it. Maybe we&#8217;ll try to add a fixed version back in someday as a potion or such.</li>
</ul>
</li>
<li>Further monster-related changes:
<ul>
<li>Demonic Crawlers lose their Warning Cry ability, their omni-elemental resistances, and some Willpower. In exchange, they get higher damage and very fast regeneration. This is intended to make Spider overall easier (by making it easier to sneak around), to reduce overlap with Swamp&#8217;s Bunyips (and their Warning Cry), and to experiment more with fast regenerating monsters.</li>
<li>Doom Hounds&#8217; Howl is now resistable by Will.</li>
<li>Many monsters (boulder beetles, two-headed ogres, various orcs&#8230;) do less damage.</li>
<li>Walking Tomes are now unable to cast while silenced.</li>
<li>Ironbound Frosthearts&#8217; Creeping Frost spell is now fully resistable by rC.</li>
</ul>
</li>
</ul>
<li>Further spell-related changes:
<ul>
<li>Manifold Assault moved to L4 (from L5) and does half as many attacks with UC as with a weapon. This is intended to make it a viable choice for more characters. Lowering its level makes it more viable for weapon-using characters to grab, and reducing its effectiveness for UC characters is compensation, since Manifold Assault was already very good for them.</li>
<li>Ozocubu&#8217;s Refrigeration&#8217;s damage has been increased.</li>
<li>Lesser Beckoning is now better at low power.</li>
<li>Slow no longer has bonus hex power.</li>
<li>The Simulacrum and Static Discharge spells have been weakened.</li>
<li>Conjure Flame and Corpse Rot have been removed. Neither were beloved by players, despite having gone through many rounds of revisions and (at least in the former case) being fairly strong; Kindle Blastsparks and Dreadful Rot are the next generation.</li>
</ul>
</li>
</ul>
<li>Interface-related changes:
<ul>
<li>Damage ratings are now shown in @. That includes ratings for unarmed combat, which were previously unavailable.</li>
<li>Bumping a foe while holding a ranged weapon now automatically shoots instead of asking if you want to hit the foe over the head with your bow.</li>
<li>Unrands have a distinct colour (teal) by default.</li>
<li>The % screen no longer shows .s for Stealth values, in an attempt to avoid misleading players into believing that there was a &#8216;maximum stealth&#8217; value that could be achieved. (That is, it now shows &#8220;Stealth: +++&#8221; instead of &#8220;Stealth: +++&#8230;&#8230;.&#8221;.)</li>
<li>Spells&#8217; maximum damage is now displayed in their descriptions, in an attempt to make it a little easier to understand how spellpower relates to spell damage</li>
<li>Many weapon brands now provide more quantitative damage information.</li>
<li>New show_blood option allows disabling blood spatters and bloody corpses.</li>
</ul>
</li>
</ul>
<li>Notable bugfixes:
<ul>
<li>Various webtiles spectator issues have been fixed.</li>
<li>The Autumn Katana no longer claims to be vorpal. (Vorpal no longer exists, but also, the Autumn Katana hadn&#8217;t *actually* been vorpal since 0.27.)</li>
<li>It&#8217;s probably no longer possible to generate an Orc:1 with zero monsters.</li>
<li>Monsters are no longer described as being both resistant and immune to blinding.</li>
<li>Monsters with &#8216;see invisible&#8217; no longer pretend to be unable to see invisible players while casting spells.</li>
<li>Monsters no longer get huge SH from orbs.</li>
<li>Monsters can no longer riposte when the player blocks their attacks.</li>
<li>Guardian golems and ironbound preservers no longer protect orbs of destruction.</li>
<li>Fsim (the wizmode fight simulator) once again works with ranged weapons.</li>
<li>Targeting a mirror cloned monster with Vampiric Draining will no longer give away its secret.</li>
<li>Jiyva&#8217;s missile-eating jelly mutation now works, for (possibly) the first time since 2014.</li>
<li>Remove Wu Jian&#8217;s secret ability to pick up mimic runes and shop at mimic shops.</li>
<li>Felids can no longer <a href="https://github.com/crawl/crawl/commit/9e567866badc7cfc8ca387d2bf0da53c9c40d3d4">throw javelins</a>.</li>
<li>Trees can no longer <a href="https://github.com/crawl/crawl/commit/95c0257040705e0654388bbbe4c497dc2949d88e">drown</a>.</li>
</ul>
</li>
</ul>
<p>Many thanks also to Sastreii for contributing many new tiles over this period, including spell icons, artefact art, new draconian sprites, and more.</p>
<p>Thanks to all the community members who&#8217;ve contributed over the last 6 months: robertxgray, Zhang Kai, Monkooky, 张凯, pdpol, Roadster Tracker, Rytis Petronis, Perry Fraser, scarf, bfaires, Aliscans, mainiacjoe, Benjamin S Wolf, Ernest Antanishinm, yrdzrfxndfvh, Alex, RypoFalem, Samantha Tobias, Michael Del Monte, Marcelo Henrique Cerri, Josh Braendel, Duong Hoang, and kd7uiy.</p>
<p>To send you off, here&#8217;s another new splash screen, this one from Micah C:</p>
<p><img src="https://github.com/crawl/crawl/raw/master/crawl-ref/source/dat/tiles/title_micah_c_ereshkigal.png" alt="Ereshkigal, by Micah C." /></p>
<p>Happy crawling!</p>


guid: http://crawl.develz.org/wordpress/?p=5242
title: Trunk Updates 1 April 2023 and Tournament Announcement
author: PleasingFungus
url: https://crawl.develz.org/wordpress/trunk-updates-1-april-2023-and-tournament-announcement
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Hello crawlers, and welcome to the most completely serious and truthful trunk update of the year! The 0.30 development cycle is (finally!) nearing a close, and we are pleased to announce a <em>tentative tournament start date</em> of <strong>Friday, May 5, 2023, at 20:00 UTC</strong><em> </em>for the sixteen day 0.30 Release tournament. We&#8217;ll shift towards bug fixes and balance as we get closer to 0.30&#8242;s release, sometime before the start of the tournament. More details about the tournament, including a tournament website and rules changes, will be available here!</p>
<p>We&#8217;d also like to celebrate the addition of two (!) new devs to the team. Félix Medrano (née robertxgray) and Implojin have both more than proven themselves by their work so far, with Félix specializing on the Crawl Android port and Implojin contributing great designs like Rampage and the toga of Victory. We&#8217;re very happy to have them onboard!</p>
<p>Anyway, let&#8217;s get some very real and serious changes headed your way.</p>
<ul>
<li>Branches and Environments:
<ul>
<li>You can now enter the Vaults without holding a rune. However&#8230; you can never leave! Unless you get a rune while you&#8217;re inside.</li>
<li>Slime walls now corrode instead of dealing damage. Unless you use Oozemancy, in which case they still do damage. Also, they don&#8217;t corrode monsters. Unless the monsters take Oozemancy damage. It&#8217;s all pretty straightforward.</li>
<li>Thunderstorms now do much more damage. If you were in the habit of standing in the middle of thunderstorms before&#8230; this might be a good time to break the habit.</li>
<li>Zigsprint now lets you start with orbs of guile or mayhem. But why would you not use Gong? It&#8217;s Gong!</li>
<li>By overwhelming popular demand, you can now place up to 100 waypoints. We know you, the players, were suffocating under the previous 10 waypoint limit. Now you are freed. Go forth!</li>
</ul>
</li>
<li>Characters:
<ul>
<li>Harm now stacks. This is a game changer.</li>
<li>Channeling effects (from Wucad Mu or orbs of energy) no longer backlash and no longer scale with Evocations. They just give -Wiz and have a flat chance of refunding spell costs. We&#8217;ve reworked this mechanic about four times this year and I&#8217;m sure it&#8217;s going to stick at some point.</li>
<li>The acidic bite mutation works again, for the first time since 2016. Oops.</li>
<li>Armataurs no longer claim to drink ten potions at once. Their tongues are long, but they aren&#8217;t actually that long.</li>
</ul>
</li>
<li>Gods:
<ul>
<li>Piety costs are now shown more clearly for invocations. Did you know that Enter the Abyss costs Piety&#8212;&#8212;-? Sort of unclear what that means, but it seems like an awful lot of -s, huh?</li>
<li>Kikubaaqudgha&#8217;s second gift is now a little less predictable. It could be anything. It&#8217;s probably about four Necromancy spells&#8230; but what if it&#8217;s not?</li>
<li>Good gods will no longer get mad when you generate negative energy clouds with a condenser vane. Instead, they&#8217;ll get even. They put their metaphysical foot down and squash the evil clouds &#8211; just like that!</li>
<li>Hepliaklqana&#8217;s Knights are now as good at attacking as ogres, or at least as slow as them.</li>
<li>Xom is no longer fixated on giving you a few different mutations.</li>
<li>Ashenzari&#8217;s trap detection ability is now only available at 4* of piety. Except that traps are always revealed these days, so this is more of a malevolence suppression ability. The world is so cruel&#8230;</li>
<li>Ashenzari&#8217;s skill boost is a little less boosted.</li>
<li>For Gozag&#8217;s gold distraction aura duration, size really doesn&#8217;t matter. (Anymore.)</li>
<li>Ru can no longer redirect butterflies&#8217; attacks.</li>
<li>Flame Tongue is removed.</li>
</ul>
</li>
<li>Unrands:
<ul>
<li>New unrand: the toga &#8216;Victory&#8217;. Gives +AC, Int, and Slay if you kill scary stuff while wearing a toga. If you drink or read anything while foes are in sight, though, you are a coward and a fool, and the toga will revert to its primal state. Victory!</li>
<li>New unrand: the Consecrated Labrys. It&#8217;s like the obsidian axe, but it&#8217;s good instead of evil, and it gets more enchanted the more scary enemies are in sight. As with the obsidian axe, we strongly recommend charging headfirst at every group of enemies while holding this.</li>
<li>New unrand: the Storm Queen&#8217;s Shield. You have probably never in your life thought &#8220;I would like to be a shock serpent&#8221;, but that&#8217;s how powerful this shield is: it will grant a wish you didn&#8217;t even know you had.</li>
<li>The Staff of Dispater is now an orb. It has +AC and *Corrode. Much to ponder&#8230;</li>
<li>The Slick Slippers are even slicker and slipperier than before. Also, they&#8217;ll no longer hustle you around when your feet are already occupied climbing stairs.</li>
<li>Majin-Bo&#8217;s spell-vamprisim now can suck up blood from a Polar Vortex. It all just flies right into the middle, it&#8217;s kinda gross.</li>
<li>Many unrands now have helpful inscriptions for their special effects, like EVERVAMP, MEGAVAMP, and probably some other things that don&#8217;t involve VAMP.</li>
<li>Morg has been removed. Actually, for all you know, it might have been removed years ago. Would you have even noticed?</li>
</ul>
</li>
<li>Items:
<ul>
<li>Orbs, scarfs, and even magical staffs can now all generate as random artefacts. Finally, you can find the orb of &#8220;Zot&#8221; of rage!</li>
<li>Randarts are very slightly, ever so imperceptibly, worse. Yes, that <b>is</b> why your best character splatted the other day. How did you know?</li>
<li>Hyperinflation has hit throwing weapons, causing javelins, boomerangs, darts, and even large rocks to skyrocket in price.</li>
<li>Tremorstones and condenser vanes are now in a set: you only get one of the two per game. Star-crossed lovers&#8230;</li>
<li>When throwing weapons break, they now deal extra damage. It&#8217;s a considerate thing. They&#8217;re very sorry that you can&#8217;t throw them around anymore and they&#8217;d like to offer you a little parting gift as they go.</li>
<li>Spectral weapons now deal damage as if you were launching an extra attack, rather than using a secret and inscrutable set of spectral weapon monster stats. They&#8217;re also not allowed to attack themselves anymore, since that was really creeping us out.</li>
<li>Longbows, arbalests, and hand crossbows have all had their stats shuffled around a little.</li>
</ul>
</li>
<li>Spells:
<ul>
<li>New spell: Plasma Beam (L5 Fire/Air), which shoots a bolt of lightning AND a bolt of fire at the most distant foe around. My first email address involved both &#8220;fire&#8221; and &#8220;plasma&#8221; so I&#8217;m very excited about this.</li>
<li>Lesser Beckoning is now level 2 (was level 3). This gives more potential design space for Greater Beckoning, to say nothing of the long-awaited Greatest Beckoning.</li>
<li>Battlespheres now fire when you cast any destructive spell, matching Vehumet. Vehumet says that a god can&#8217;t be matched by a mere L5 spell, but I&#8217;m unconvinced.</li>
<li>Arcjolt can now hit any monster in sight, as long as there are a continuous chain of other monsters between you and that monster.</li>
<li>Irradiate now only malmutates monsters sometimes. Try saying &#8216;pretty please&#8217; before casting it for best effects.</li>
<li>Lightning Bolt is gone. Amazing it lasted this long, really. Not really famous for their longevity, lightning bolts.</li>
</ul>
</li>
<li>Monsters:
<ul>
<li>New monster: sleepcaps. If they hit you, their spores might put you to sleep. If they put you to sleep, the next thing that hits you will get a big stabbing damage multiplier. And if you lose all your hit points? That&#8217;s right: you die in real life.</li>
<li>New monster: skysharks, cutting through the air (as sharks are known to). If they bite you and draw blood, somehow <i>they&#8217;re</i> the ones that get mad about it? And that makes them bite even harder? Seems very unfair, frankly.</li>
<li>Smitten monsters now wake up, but they&#8217;re not happy about it.</li>
<li>Giant fish zombies have been removed.</li>
<li>Endotherms have been removed.</li>
</ul>
</li>
<li>Webtiles:
<ul>
<li>You can now enter explore mode online! Hit + to explore freely&#8230; without any fear of death. You are immortal. You are a god!</li>
<li>You can now block trolls. They cannot regenerate from this. (Not yet enabled on every server: this will roll out to servers next time they restart webtiles.)</li>
</ul>
</li>
<li>Crawl has, somehow, become even more Australian.</li>
<li>We fixed a typo in which we wrote &#8220;entire the&#8221; instead of &#8220;the entire&#8221;. Not really a big deal, honestly&#8230; except that this text has been in every single morgue since 2007! Shouldn&#8217;t someone have told us? We&#8217;re so embarrassed right now.</li>
<li>Last but not least, we rewrote maybe_bool to be, arguably, better. See below for<br />
motivation, but here are the practical notes up front:</p>
<pre>* `maybe_bool` becomes a class rather than an enum.
* After this commit, nearly anywhere you would write MB_FALSE or
  MB_TRUE you would simply write `false` or `true`.
* Where you would write MB_MAYBE you now should write
  `maybe_bool::maybe`.
* Comparison to `true` or `false` is safe and will do exact
  comparison. (The latter may appear surprising, but it is because
  comparison will cause a bool to be cast to maybe_bool, rather than a
  maybe_bool to be cast down to bool.)
* Semantics change: `maybe_bool::maybe` does not convert to true by
  default under any circumstance.
* In cases where you would have previously written `mb == MB_TRUE` you
  can just rely on a bool cast (`maybe` does not convert to `true`). In
  cases where you would write `mb == MB_FALSE` you can write `!mb`.
* frombool and tobool are replaced respectively by constructors and
  the class method maybe_bool::to_bool (as well as an explicit bool
  cast).
* While bool explicit operators have weird rules and can be used
  implicitly in some cases, there are still times when you'll need
  to do the cast. In particular, if a function returns a bool, to
  return a maybe_bool you'll need to explicitly cast.
* Mixing bools and maybe_bools in logical expressions generally requires
  an explicit cast. (If you write a condition that just uses !, that
  doesn't need one.)

Old `maybe_bool` had the advantage of simplicity: it was just an enum
and as long as these enum values are used consistently, it is pretty
simple to use and as efficient as any integer-based type, with
effectively no implementation code.

However, there are a few complaints about it. First, the naming scheme
was quite ad hoc across the enum values and various functions that used
them.  Second, and more importantly in my mind, the enum-based
implementation led in a very non-clean way to implicit int and bool
casts, and a very ad hoc semantics for this type, especially in its
interaction with boolean expressions. Taken as a three-valued logic, it
is very weird. The law of the excluded middle *is* valid (in the sense
that `p || !p` is always true), but double negation elimination is *not*
valid (i.e. `!!p != p`); the weaker double negation inference `!!!p ==
!p` does hold conceivably classing it with some paraconsistent logics,
though I haven't found an exact analogue (see [1], though this logic
doesn't much resemble L8 there, which has the same double negation
pattern). At a practical level this ad hoc nature made mixing with bools
in both directions weird and somewhat error prone; aside from the above
concerns (which essentially result from MB_MAYBE implicitly casting to
`true`), `true` would cast to the enum value MB_MAYBE, leading to clunky
conversion code in a lot of places, needing to always use MB
constants, and error-prone comparison with regular bools.

This commit tries to systematize things more cleanly, albeit with a much
more complicated implementation. The underlying data type does still
amount to a fancy enum (now an enum class), but it's wrapped with a more
clearly defined class, largely inspired by boost's logic::tribool.  (I
considered whether importing one of the various "`optional` but for
c++11" packages out there might be a solution, but ultimately rejected
this, because these classes tend to have an even weirder, for a
bool-based type, set of implicit conversions to bool). A `bool`
constructor is provided making it generally safe to use `true` and
`false` as e.g. return values, and an explicit cast to bool is provided
that has a better default than MB_MAYBE->true. (In some cases, this may
be the desired semantics, of course, but this is still doable via
`to_bool` or direct checks against false.) The operator semantics is
based on the standard "Strong Kleene" three-valued logic operators,
which provide a (imo) very sensible set of defaults for a three-valued
logic. Note that because of the existence of the bool operator, this
won't let you mix `maybe_bool`s in logical expressions together with
bools without some explicit casting one way or the other.  The
implementation is nearly (but not quite) purely header-based.

As part of this commit, I have implemented everything as a
back-compatibility layer and purely just tried to clarify the semantics
of various instances of maybe_bool in cases where it did get an implicit
conversion. The bulk of this is around `can_wear_item` which does tend
to treat its maybe return as truth-y.  A future commit will convert
MB_TRUE etc into their `maybe_bool::t` etc equivalents.

[1] Kamide 2013, A Hierarchy of Weak Double Negations, Studia Logica 101.6</pre>
</li>
</ul>
<p>Many thanks also to Sastreii and CanOfWorms for contributing many new tiles over this period, including spell icons, artefact art, Shrek, and more.</p>
<p>Thanks also to all the community members who&#8217;ve contributed over the last few months, who I am too tired to credit in full. In particular, check out what SpinningBird sent us!</p>
<p><img src="https://raw.githubusercontent.com/crawl/crawl/33407759773a21cb858ef343666b2cff6101964f/crawl-ref/source/dat/tiles/title_SpinningBird_djinn_sears_gnolls.png"/></p>
<p>Happy crawling! Stay serious out there.</p>


guid: http://crawl.develz.org/wordpress/?p=5252
title: 0.30: “The Reavers Return”
author: gammafunk
url: https://crawl.develz.org/wordpress/0-30-the-reavers-return
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We are pleased to announce the release of Dungeon Crawl: Stone Soup 0.30 “The Reavers Return”!</p>
<p>DCSS 0.30 features the return of the legendary Reaver background. These warrior-mages start with two new powerful yet situational spells: Kiss of Death and Momentum Strike. They also start with the Hailstorm spell, which has been moved down to level 3 yet can sometimes miss.</p>
<p>The new Armataur species replaces Palentongas, who have rolled peacefully off into the sunset. Armataurs are large, scaly mammals that wear bardings, but instead of rolling and curling, they have innate rampage and a long tongue mutation that doubles potion effects and slows down scroll reading.</p>
<p>A grand total of 7 new spells have been added! Two we&#8217;ve already mentioned for Reaver, with two more being likewise new designs and three replacing existing spells:</p>
<p style="padding-left: 30px;">- <em>Kiss of Death</em> (L1 Conj/Necro): does high draining damage to an adjacent enemy, at the cost of significant max HP drain to the caster.<br />
- <em>Momentum Strike</em> (L2 Conj/Tloc). Redirects the caster&#8217;s momentum for a damaging attack that doesn&#8217;t require line of fire, but locks the caster in place for some turns after a hit (and cannot be recast until this expires).<br />
- <em>Cigotuvi&#8217;s Dreadful Rot </em>(L2 Necro/Air/Pois). Rots away the caster&#8217;s flesh to create a cloud of deadly miasma on their tile. Replaces Corpse Rot.<br />
- <em>Volatile Blastmotes</em> (L3 Fire/Tloc). Creates an explosive cloud on the caster&#8217;s tile, usable as a land mine or tricky tool. Replaces Conjure Flame.<br />
- <em>Vhi&#8217;s Electric Charge</em> (L3 Tloc/Air). Launches the caster at a nearby foe and launches a crackling melee attack, phasing through creatures en route.<br />
- <em>Arcjolt</em> (L5 Air/Conj). Electrocutes everything adjacent to the caster, everything adjacent to those creatures, etc, etc, in a chain. Replaces Lightning Bolt.<br />
- <em>Plasma Beam</em> (L6 Fire/Air): Fires a bolt of lightning and another bolt of fire, both at the most distant foe.</p>
<p>We&#8217;ve added 6 new unrands and made changes to 11 existing ones. Additionally you can now find randart orbs, scarves, and magical staves!</p>
<p>Finally, Lugonu has been mutated into a new form. Banishment is available at 2* piety (instead of 3*) and mutates foes that resist getting banished. To encourage adventurers to abandon their current god, Lugonu helpfully banishes some hostile monsters whenever other gods act with their wrath. There are also now more Lugonu altars in the Abyss, and more piety is awarded for conversions in the Abyss.</p>
<p>Download DCSS 0.30 <a title="download" href="http://crawl.develz.org/download.htm">here</a> or <a title="play online" href="http://crawl.develz.org/play.htm">play it online</a> on one of many servers across the world! Packages for Windows and OS X are available now. Linux debs are coming soon. See the full list of changes on the release page <a href="https://github.com/crawl/crawl/releases/tag/0.30.0">here</a>.</p>
<p>The tournament starts on <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20230505T20">Friday May 5th 8pm UTC</a> and runs through <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20230521T20">Sunday May 21st 8pm UTC</a>, with all online 0.30 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.30/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until <a href="https://www.timeanddate.com/worldclock/fixedtime.html?iso=20230512T20">Friday May 12th 8pm UTC</a>.</p>
<p>Many thanks to all those who have <a title="credits" href="https://raw.githubusercontent.com/crawl/crawl/master/crawl-ref/CREDITS.txt">contributed</a> to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.30 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.30 and a warm welcome to the first-time contributors, listed in italics. (Let us know if someone is missing, this data is tricky to collect in bulk!)</p>
<blockquote><p>Aliscans, Ernest Antanishin, Antern, bfaires, <em>Josh Braendel</em>, Marcelo Henrique Cerri, Perry Fraser, <em>greenscarf</em>, <em>Duong Hoang</em>, Alex Jurkiewicz, kippig, <em>kd7uiy</em>, mainiacjoe, Monkooky, Michael Del Monte, pdpol, Brian Power, Roadster Tracker, Rypofalem, Rytis Petronis, <em>Aleksando Sansan</em>, sdynet, Skrybe, <em>Antal Spector-Zabusky</em>, <em>spiros</em>, Sergio Thompson, Samantha Tobias, Benjamin S Wolf, <em>yrdzrfxndfvh</em>, Zhang Kai</p></blockquote>


guid: http://crawl.develz.org/wordpress/?p=5277
title: 0.30 Tournament Results
author: gammafunk
url: https://crawl.develz.org/wordpress/0-30-tournament-results
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The <a title="0.30 tournament" href="https://crawl.develz.org/tournament/0.30/overview.html">0.30 tournament</a> is over. On behalf of the Dev Team, many thanks to all the server admins, outside contributors, bug reporters, and to the many DCSS players who made the 0.30 release and tournament possible!</p>
<p>For 16 days, players could compete for tournament points and banners by playing 0.30 games on the public servers. This was the fourth tournament held after the major revision of the tournament rules done for 0.25. There were changes based on previous input, and once again we&#8217;ve collected the feedback posted on tavern and reddit.</p>
<p>Congratulations to <a href="https://crawl.develz.org/tournament/0.30/players/yegres.html">Sergey</a> (aka yegreS),  for once again taking first place in the <a href="https://crawl.develz.org/tournament/0.30/all-players-ranks.html">individual competition</a>, with 87,793 points! Over the course of the tournament, Sergey won all 11 games he played (!), the 3rd longest win streak overall. He also achieved an 86M <a href="https://underhound.eu/crawl/morgue/yegreS/morgue-yegreS-20230517-165837.txt">MeIE</a> high score (5th highest overall), a <a href="http://crawl.xtahua.com/crawl/morgue/yegreS/morgue-yegreS-20230507-122025.txt">0:30:53</a> realtime win (6th best overall), collected 14 tier 3 banners, got a species and a background high score (from a 45M score <a href="https://underhound.eu/crawl/morgue/yegreS/morgue-yegreS-20230513-002307.txt">GnEE</a>), won 4 Nemelex&#8217;s Choice combos, got 4 combo high scores, and completed 27 ziggurats.</p>
<p>This is the first time someone has won the tournament with 100% winrate since <a href="http://crawl.akrasiac.org/tourney12b/players/theglow.html">theglow</a> won all 18 of their games in the <a href="http://crawl.akrasiac.org/tourney12b/overview.html">0.11 tournament</a> in 2012. <a href="https://crawl.develz.org/tournament/0.30/players/elliptic.html">elliptic</a> (27th(!) overall) notes that there have been 24 tournaments with the 16-day format, and in the first 12 of them there were 15 distinct players who won at least 18 games in a single tournament, while in the second 12 of them there were 66 such players. Despite this trend, Sergey, who also took 1st place in the previous 0.29 tournament, took 1st place in this tournament with 14 fewer wins (11 vs 25 in 0.29) and played over 3 hours less per day on average (4:43:25 vs 7:58:28 in 0.29)!</p>
<p>Other big winners:</p>
<ul>
<li><a href="https://crawl.develz.org/tournament/0.30/players/kuniqs.html">kuniqs</a>: 2nd place overall (82,079 points, 19 wins), with a 96M <a href="https://underhound.eu/crawl/morgue/kuniqs/morgue-kuniqs-20230520-144825.txt">MeEE</a> high score (2nd overall), a <a href="https://underhound.eu/crawl/morgue/kuniqs/morgue-kuniqs-20230521-084049.txt">0:29:35</a> realtime win (4th overall), 18 tier 3 banners (4th most overall), a 5 win streak, 7 Nemelex&#8217;s Choice wins, two combo high scores, and 28 Ziggurats cleared.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/shadowrider38.html">ShadowRider38</a>: 3rd place overall (77,239 points, 21 wins), with a <a href="https://crawl.kelbi.org/crawl/morgue/ShadowRider38/morgue-ShadowRider38-20230515-231720.txt">0:35:02</a> realtime win (7th overall), a 17,829 turns <a href="https://crawl.kelbi.org/crawl/morgue/ShadowRider38/morgue-ShadowRider38-20230515-055221.txt">MeFi</a> win (8th lowest overall), 15 tier 3 banners (5th most overall), 8 Nemelex&#8217;s Choice wins (9th overall), 11 combo high scores, a 5 win streak, and 27 Ziggurats cleared.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/narz.html">narz</a>: Highest scoring win at 132M with <a href="https://crawl.kelbi.org/crawl/morgue/narz/morgue-narz-20230516-030420.txt">MeEE</a> and the most pacific win with a <a href="https://crawl.kelbi.org/crawl/morgue/narz/morgue-narz-20230517-200309.txt">SpEn</a> that killed only 186 monsters.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/flugkiller.html">FlugKiller</a>: Fastest realtime win at 00:21:48 with a <a href="https://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20230509-123229.txt">MiFi</a>.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/themeinteam.html">TheMeInTeam</a>: Longest win streak with 15 wins.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/wizard1ke.html">Wizard1ke</a>: The only person to collect all 24 tier 3 banners.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/removed.html">removed</a>: 30 Nemelex&#8217;s Choice wins.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/fizzlegong.html">FizzleGong</a>: Lowest turncount win with a <a href="https://cbro.berotato.org/morgue/FizzleGong/morgue-FizzleGong-20230512-202410.txt">MeEE</a> winning in 5,881 turns.</li>
<li><a href="https://crawl.develz.org/tournament/0.30/players/snakkusnakku7.html">SnakkuSnakku7</a>: 3 species (At, Gh, HO) and 3 background (Ar, CK, Hs) high scores, and 1st in the Combo High Scores category.</li>
<li>The first three wins of the tournament were by <a href="https://crawl.develz.org/tournament/0.30/players/enegeticocto.html">EnegeticOcto</a> (a <a href="http://crawl.xtahua.com/crawl/morgue/EnegeticOcto/morgue-EnegeticOcto-20230505-205706.txt">MiFi</a> in 0:56:35), ShadowRider38 (a <a href="https://crawl.kelbi.org/crawl/morgue/ShadowRider38/morgue-ShadowRider38-20230505-211006.txt">MiFi</a> in 1:08:58), and <a href="https://crawl.develz.org/tournament/0.30/players/hihot.html">hihot</a> (a <a href="http://crawl.akrasiac.org/rawdata/hihot/morgue-hihot-20230505-213818.txt">MiFi</a> in 1:31:43).</li>
<li>The last win of the tournament was by <a href="https://crawl.develz.org/tournament/0.30/players/railbird80.html">RailBird80</a>, a <a href="http://crawl.akrasiac.org/rawdata/Railbird80/morgue-Railbird80-20230521-194439.txt">HuFE</a> that ascended 16 minutes before the end of tournament yet took over 4 days to win.</li>
</ul>
<p>The <a href="https://crawl.develz.org/tournament/0.30/teams.html">clan competition</a> was won by <a href="https://crawl.develz.org/tournament/0.30/clans/heavy-weight-lifting-club-enegeticocto.html">Heavy Weight Lifting Club</a> (captained by EnegeticOcto) with 112,811 points. They sealed their victory by besting the 2nd place clan in the Combo High Scores (1st overall), Most Pacific Win (1st overall), Fastest Realtime (1st overall), Piety, Nemelex&#8217;s Choice, and Streak categories. In a close 2nd place was <a href="https://crawl.develz.org/tournament/0.30/clans/uncle-xoms-tabbin-themargong.html">Uncle Xoms Tabbin</a> (captained by <a href="https://crawl.develz.org/tournament/0.30/players/themargong.html">themargong</a>) with 104,283 points, and first place in the High Score and Lowest Turncount Win categories. They tied for 1st place with Heavy Weight Lifting Club in the banners category, together collecting all 24 tier 3 banners. 3rd place was held by <a href="https://crawl.develz.org/tournament/0.30/clans/my-boolean-is-binary-themeinteam.html">My Boolean is Binary</a> (captained by TheMeInTeam) with 93,023 points, and 1st place in the Nemelex&#8217;s Choice and Streak categories.</p>
<p>To close out, here are some overall tournament statistics (with the 0.29 numbers in brackets for comparison). 2,765 players started a game at some point during the tournament; of those 2,711 completed a game without quitting or leaving the dungeon sadly orbless. This definition of “player” (players who completed a non-boring game) is used for subsequent calculations and was used for the 0.29 statistics.</p>
<ul>
<li>Players: 2,711 (0.29: 2,573)</li>
<li>Total player time: 30,408 hrs (0.29: 27,360 hrs)</li>
<li>Average player time: 11.2 hrs (0.29: 10.6 hrs)</li>
<li>Games played: 69,601 (0.29: 64,339)</li>
<li>Players who got a rune: 926 (0.29: 759)</li>
<li>454 winners and 1433 wins (0.29: 368 winners and 1183 wins)</li>
<li>Win rate: 2.06% (0.29: 1.84%)</li>
<li>Proportion of players using WebTiles: 97.1% (0.29: 96.9%)</li>
</ul>
<p>Thanks to you all for playing! Many thanks as well to server admins, developers, and contributors who made the 0.30 release and tournament possible!</p>
<p>The 0.31 season has begun, and changes are already in motion. Watch this space for further information as the 0.31 trunk update posts begin to flow. Until next time, happy crawling!</p>


guid: http://crawl.develz.org/wordpress/?p=5310
title: Trunk Updates 11 December 2023 and Tournament Announcement
author: PleasingFungus
url: https://crawl.develz.org/wordpress/trunk-updates-11-december-2023-and-tournament-announcement
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Hello crawlers, and welcome to the ultimate trunk update of the year! The 0.31 development cycle is (finally!) nearing a close, and we are pleased to announce a <em>tentative tournament start date</em> of <strong>Friday, Jan 19, 2024, at 20:00 UTC</strong><em> </em>for the sixteen day 0.31 Release tournament. We&#8217;ll shift towards bug fixes and balance as we get closer to 0.31&#8242;s release, sometime before the start of the tournament. More details about the tournament, including a tournament website and rules changes, will be available here!</p>
<p>A heads-up also that crawl.kelbi.org will be shutting down after the tournament. Many thanks to Floraline for their years of hard work maintaining the server! We&#8217;ll have news on a replacement server sometime in the coming few months.</p>
<p>Looks like we somehow have not even posted one trunk update since 0.30 released (!), so to fit all the major changes in, this post will be a little more condensed than usual. I&#8217;ll still try to sneak in one or two fun things, and the <a href="https://github.com/crawl/crawl/blob/60294ff2a06f2ed2b4598bb5236e6f861656acfa/crawl-ref/docs/changelog.txt">changelog</a> has more details on some things.</p>
<ul>
<li>Shapeshifting and Transmuting:
<ul>
<li>Form spells have turned into a new type of item, &#8216;talismans&#8217;.</li>
<li>Talismans are very rare items that, when used, transform the user into a corresponding form until they choose to leave it.</li>
<li>A new skill, Shapeshifting, governs use of talismans. Like Spellcasting, the &#8216;default&#8217; (human) aptitude for this is -1.</li>
<li>Form changes:
<ul>
<li>Beastly Appendage has been replaced by Beast Form, provides innate Slaying but melds all auxiliary armour.</li>
<li>Spider Form has been replaced by Flux Form, which causes melee attacks to magically contaminate and explode victims. It melds weapons, shields, and body armour.</li>
<li>Ice Form has been replaced by Serpent Form, which transforms the user into a massive, foe-crushing python.</li>
<li>New form: Maw, which opens a horrible, foe-devouring second mouth in the user&#8217;s chest. It grants an auxiliary bite attack and sometimes heals the user when they kill foes in melee, in exchange for melding body armour.</li>
<li>Lich Form has been replaced by Death Form, which severely debilitates foes struck in melee and grants an active &#8216;Siphon Essence&#8217; ability that torments nearby foes to heal the player. It&#8217;s otherwise similar.</li>
<li>All forms have had their stats adjusted and scaling rebalanced.</li>
</ul>
</li>
<li>A new background, Shapeshifter, replaces the Transmuter start. Shapeshifters start out with two talismans, Beast and Flux.</li>
<li>The Transmutations skill continues to support use of the ~ten remaining Transmutations spells. There are no current plans to remove it.</li>
<li>You can read more about the motivation for these changes <a href="https://github.com/crawl/crawl/wiki/Transmutations-Reform">here</a> and <a href="https://github.com/crawl/crawl/commit/18800231877e12caceb48c2f929f842d55aac934">here</a>. Note that many of the implementation details have changed since those were written.</li>
</ul>
</li>
<li>Hexslingers:
<ul>
<li>Hexslingers have had their starting spell set vigorously reworked.</li>
<li>New L2 Hex spell: Jinxbite. Deals additional Will-checking damage to foes attacked while its duration lasts, but also halves the caster&#8217;s will. Lots of faerie laughter is involved.</li>
<li>New L3 Hex spell: Sigil of Binding. Creates two magical traps nearby, which temporarily bind hapless victims in place. Don&#8217;t step in these yourself!</li>
<li>New L4 Hex spell: Dimensional Bullseye (replacing Portal Projectile). Targets a foe and teleports all shots fired at other foes to also hit the chosen unfortunate.</li>
<li>Inner Flame and Cause Fear are still included. Slow is no longer in the starting spell set.</li>
</ul>
</li>
<li>Uniques:
<ul>
<li>New: Jeremiah, a peaceful barachi trapped between sleep and waking. Their dreams leak into reality in a stream of beautiful butterflies.</li>
<li>Returned and revamped: Norris, a death cultist who surfs into battle with a gang of skysharks.</li>
<li>Maurice is now a vine stalker, with high regen and antimagic bites.</li>
<li>Erica is now an octopode, with constriction and a more focused spell set.</li>
<li>Frederick has a new spell set.</li>
</ul>
</li>
<li>Other monsters:
<ul>
<li>New and returning:
<ul>
<li>Kobold blastminers, who blast enemies with alchemical cannons and get sent flying backwards by the recoil.</li>
<li>Brain worms, which eat magic and minds alike</li>
<li>Glass eyes (replacing floating eyes), which vitrify victims to make them fragile as glass.</li>
<li>Formless jellyfish, which constrict and sting for single-turn paralysis.</li>
<li>Arcanists, a kind of fairly dangerous vitrifying wizard.</li>
<li>Crocodiles, which drag victims backward.</li>
</ul>
</li>
<li>Alligators and reapers are now somewhat weaker, but drag victims backward.</li>
<li>Ufetubi and tengu warriors now slip behind their foe before meleeing.</li>
<li>Tengu reavers now have rMsl.</li>
<li>&#8216;Wizard&#8217; has been renamed to &#8216;occulist&#8217; and is now somewhat stronger.</li>
<li>Great orbs of eyes have a new spell set, and are slightly stronger.</li>
<li>Deep Elf Elementalists now cast LRD instead of Awaken Earth.</li>
<li>Red very ugly things now do direct fire damage instead of sticky flame.</li>
<li>Abominations&#8217; speed is no longer randomized. (Did you know it was..?)</li>
<li>Behavioral changes:
<ul>
<li>Attacks of opportunity have been reworked. Previously, whenever players moved away from a same-speed-or-faster monster, the monster had a 1/3 chance of launching a free melee attack. Now, they launch such attacks only if they move to follow the player, and there&#8217;s a chance for the attack to cost a small amount of their time (so they can fall behind). The old &#8216;random energy&#8217; effect, causing monsters to have a chance of gaining or losing a small amount of time when moving, has returned specifically when monsters are pursuing the player. Some discussion of the reasoning for these changes: <a href="https://github.com/crawl/crawl/commit/e438bbe35ce7f90b92d42aa83db79bc0207f99b4">A</a>, <a href="https://github.com/crawl/crawl/commit/4209ae197b53c91ba4e1345fe5cd51a30251e854">B</a>, and <a href="https://github.com/crawl/crawl/commit/66562123206107b4e469b57e8098b9c7ca6ab157">C</a>. The last of those describes an experimental &#8216;hot pursuit&#8217; section which has since been removed, but &#8216;history&#8217; section remains applicable.</li>
<li>Giant monsters can now be ensnared by webs and nets.</li>
<li>Jellies can no longer be netted or constricted.</li>
<li>Allied undead no longer become angry when damaged.</li>
<li>Monsters no longer use up charges when zapping wands.</li>
<li>Monsters can no longer use scrolls or potions.</li>
<li>Some monsters now have special dialogue after killing the player.</li>
</ul>
</li>
</ul>
</li>
<li>Other species changes:
<ul>
<li>At XL 7, Armataurs now gain temporary regeneration when they roll (rampage) toward foes. Their base health has been decreased to compensate.</li>
<li>Felids now get an extra life at XL 1, don&#8217;t lose XLs on dying, and have somewhat more base health. They no longer have fast movement &#8211; see <a href="https://github.com/crawl/crawl/commit/406407be8511ac9846d21bd4a51990ab4ce1dbb7">here</a>.</li>
<li>Tengu now have inherent Acrobat (+15 EV after moving/waiting) at XL 1, and gain +4 EV at XL7 instead of a 20% EV bonus.</li>
<li>Mummies now get their first necromancy enhancer at XL 1, not 3.</li>
<li>Vine Stalkers now get regeneration at XL 1, not 4.</li>
<li>Djinn now use all magic skills, instead of only Spellcasting. Djinn can only choose to train all magic skills or none. This is pretty similar to how they worked before, but means that e.g. they can now use skill manuals, do elemental staff or pain brand damage, and care about other types of Ashenzari skill curses.</li>
<li>Ghouls start with an additional +4 Int, to encourage more variety in playstyles.</li>
<li>Formicids can no longer be involuntarily shafted.</li>
</ul>
</li>
<li>Items:
<ul>
<li>An old favorite returns: sack of spiders, an XP-charged evoker which webs foes and summons creepy-crawlies. Alternates with boxes of beasts.</li>
<li>Changes to ranged weapons:
<ul>
<li>New weapon: orcbow, between a shortbow and an arbalest in strength.</li>
<li>Hand crossbows are now hand cannons, alchemically belching smoke and death. (See <a href="https://github.com/crawl/crawl/commit/34c7ba284f62950488e3860eb4499ce7605a6146">here</a> for why.)</li>
<li>New unrand: Mule, a +7 flaming hand cannon which sometimes knocks the user backwards when fired.</li>
</ul>
</li>
<li>Potions of flight replaced with potions of enlightenment, which also give Will+.</li>
<li>Scrolls of magic mapping replaced with scrolls of revelation, which also give X-ray vision and sInv for one turn.</li>
<li>Quick blades now hit twice for every swing, with a higher attack delay. Gyre &#038; Gimble hits four times.</li>
<li>Artefact staves can now enhance other schools of magic, even those which don&#8217;t have corresponding base enhancer staff types.</li>
<li>The dreamshard necklace now has Acrobat, but only restores the player to 1 HP when they&#8217;d otherwise die, rather than healing 50-100% of their max HP.</li>
<li>To simplify, shields no longer degrade in effectiveness with each hit taken per-turn. Instead, they now have a limit on per-turn blocks, based on shield type &#8211; bucklers give 2, kites give 3, towers give 4.</li>
</ul>
</li>
<li>Spells:
<ul>
<li>New: Brom&#8217;s Barrelling Boulder (L4 Earth/Conj). Creates a boulder, which crushed and knocks back foes until it&#8217;s broken or it hits a wall. (In the latter case, it explodes!)</li>
<li>Arcjolt now initially arcs to foes within two tiles of the caster.</li>
<li>Static Discharge now does more damage, but checks half AC instead of none.</li>
<li>Borgnjor&#8217;s Vile Clutch (and Fastroot) now work on invisible enemies, but the former now does less damage.</li>
<li>Hailstorm does more damage at low power and less at high power.</li>
<li>Swiftness now works in water. (Did you know it didn&#8217;t?)</li>
</ul>
</li>
<li>Other Notes:
<ul>
<li>The Slime Pits now have many possible endings, instead of one fixed pre-designed one. Many thanks to long-time contributor nicolae for his work on this one!</li>
<li>Monsters with unusual items (e.g. distortion) are now highlit in purple by default.</li>
<li>Okawaru now gives exactly one weapon gift and one armour gift as a capstone (chosen by the player), rather than giving many random gifts.</li>
<li>Nemelex&#8217;s Cloud, Elements and Swarm cards have been changed to scale more appropriately with power.</li>
<li>The Ecumenical Temple is now fully mapped on entry. This saves time for some of the really big temples, and also lets us add really silly stuff like a labyrinth-themed temple. Not that we&#8217;d ever actually add one of those, of course&#8230;</li>
</ul>
</li>
</ul>
<p>Many thanks also to Sastreii and CanOfWorms for contributing many new tiles over this period, including spell icons, artefact art, cat dragons, and more. In particular, thanks to CanOfWorms for commissioning this lovely new splash screen.</p>
<p><img src="https://raw.githubusercontent.com/crawl/crawl/d88bbb8b4df8043f4d187c382662d4e0f9e6d531/crawl-ref/source/dat/tiles/title_king7artist_eustachio.png"/></p>
<p>Thanks also to all the community members who&#8217;ve contributed over the last year, who I am too tired to credit in full. Crawl wouldn&#8217;t be Stone Soup without you.</p>
<p>Happy crawling, and happy holidays!</p>


guid: http://crawl.develz.org/wordpress/?p=5352
title: 0.31 Tournament Page
author: gammafunk
url: https://crawl.develz.org/wordpress/0-31-tournament-page
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Hello crawlers! We&#8217;re letting you know that the <a href="https://crawl.develz.org/tournament/0.31/">official 0.31 tournament page</a> is online! The will run from <em>Friday January 19th, 8pm UTC to Sunday February 4th, 8pm UTC</em>. Please bookmark the 0.31 tournament page and use this page to track your progress throughout the tournament. We have some rule changes planned and will update the tournament page with any changes. As we test the tournament scripts, you may see summaries of test data in the &#8220;Overview&#8221; page taken from recent 0.31 beta games. Don&#8217;t worry, we&#8217;ll reset everything before the tournament starts.</p>
<p>The <em>0.31 release</em> which will be made and announced either this Thursday or Friday. In the meantime, the 0.31 branch is being rolled out to our online servers. Servers CBR2, CKO, and LLD all have it available, with more to come soon! You can <a href="https://crawl.develz.org/tournament/0.31/#how-to-play">configure your tournament clan</a> in your RC files right now on one of those three servers, or simply wait until your favorite official server has 0.31 installed. Finally, we&#8217;re finishing up one last belated Trunk Update to fill you in on the torrent of changes we had in December. So stay tuned, and good luck in the tournament!</p>


guid: http://crawl.develz.org/wordpress/?p=5368
title: 0.31 “The Alchemy of Forms”
author: gammafunk
url: https://crawl.develz.org/wordpress/0-31-the-alchemy-of-forms
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We are pleased to announce the release of Dungeon Crawl: Stone Soup 0.31 “The Alchemy of Forms”!</p>
<p>DCSS 0.31 features some major spell school and starting background changes, a variety of new spells, many dangerous new monsters, and some major species changes.</p>
<p>The Transmutations forms spells have been moved into talisman items, which allow permanently shifting into forms based on the new Shapeshifting skill. The remaining Transmutations spells and the Poison Magic spells have been merged into a new Alchemy school. The Shapeshifter and Alchemist starting backgrounds replace Transmuter and Venom Mage, respectively. To support these changes, this release adds 4 new spells, sees 7 spells reworked into improved replacements, and has many tweaks to the levels and schools. In fact it&#8217;s a bit too much for me to write up as we&#8217;re getting ready for tournament! So please check out the &#8220;Spells&#8221; section of the <a href="https://github.com/crawl/crawl/releases/tag/0.31.0">release</a> for full details.</p>
<p>The new Oni species replaces Ogres. Oni love a good drunken brawl, gaining double effect when drinking any potion that restore HP or MP, and also perform an immediate cleaving melee attack. They are otherwise similar to Ogres, but also have horns and some increased apts. Starting at XL 7, Armataurs now regenerate HP and MP while rampaging. They also lose their double potion effect, and no longer read scrolls slowly. Felids now have an extra life at XL 1 and don&#8217;t lose XLs when dying. Tengu have an inherent acrobat effect (like the amulet) at XL 1, and gain 4 EV at XL 7 instead of gaining an EV bonus and fast movement. Humans gain the exploration healing effect formerly found on Meteorans, which are no more. Instead, players of any species can now rush to find gems stashed at the end of many branches as achievement items.</p>
<p>Download DCSS 0.31 <a href="https://crawl.develz.org/download.htm">here</a> or play it online on one of many servers across the world! Packages for Windows, OS X, and Linux are available now. See the full list of changes on the release page <a href="https://github.com/crawl/crawl/releases/tag/0.31.0">here</a>.</p>
<p>The tournament starts on Friday January 19th 8pm UTC and runs through Sunday February 4th 8pm UTC, with all online 0.31 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.31/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until Friday January 26th 8pm UTC.</p>
<p>Many thanks to all those who have contributed to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.31 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.31 and a warm welcome to the first-time contributors. (Let us know if someone is missing, this data is tricky to collect in bulk!)</p>
<blockquote><p>910rd, aedyr, AlexanderPosch, Aliscans, Alvin, Anabel, Antal Spector-Zabusky, CipHuK, cureja, DevJac, geekosaur, Jared Miller, kippig, laserbat, MainiacJoe, mgdelmonte, Monkooky, Nephila Hirsch Oliveira, NormalPerson7, orjb1, Perren Yang, Quinten Konyn, Roadster Tracker, Rosstin Murphy, RypoFalem, Rytis Petronis, sandertyu, sdynet, Sean Dewar, Sebastian Zivota, Skrybe, yrdzrfxndfvh, Zhang Kai</p></blockquote>


guid: https://crawl.develz.org/wordpress/?p=5399
title: 0.31 Tournament Results
author: gammafunk
url: https://crawl.develz.org/wordpress/0-31-tournament-results
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The <a title="0.31 tournament" href="https://crawl.develz.org/tournament/0.31/overview.html">0.31 tournament</a> is over. On behalf of the Dev Team, many thanks to all the server admins, outside contributors, bug reporters, and to the many DCSS players who made the 0.31 release and tournament possible!</p>
<p>For 16 days, players could compete for tournament points and banners by playing 0.31 games on the public servers. The only major rule changes compared to the 0.30 tournament were the addition of a Gem Collection category, making Uskayaw&#8217;s banner be based on gems, and allowing Felids to win the Ruthless Efficiency banner.</p>
<p>Congratulations to <a href="https://crawl.develz.org/tournament/0.31/players/sergey.html">Sergey</a>,  for once again taking first place in the <a href="https://crawl.develz.org/tournament/0.31/all-players-ranks.html">individual competition</a>, with 115,521 points and 16 wins! Sergey got the most points for combos won, with species high scores for HO and Hu, and class scores for Hu and Wn, won the most pacific game (a <a href="http://crawl.xtahua.com/crawl/morgue/Sergey/morgue-Sergey-20240123-185019.txt">SpEn</a> with only 421 kills), achieved an 8 win streak (3rd best overall), a 64M <a href="http://crawl.xtahua.com/crawl/morgue/Sergey/morgue-Sergey-20240121-170932.txt">HuEE</a> high score (3rd best overall), collected 17 tier 3 banners, and won 6 Nemelex&#8217;s Choice combos.</p>
<p>Other big winners:</p>
<ul>
<li><a href="https://crawl.develz.org/tournament/0.31/players/wizard1ke.html">Wizard1ke</a>: 2nd place overall with 106,191 points and 28 wins. Wizard1ke was the only player to collect all 24 tier 3 banners, and had the most points in the piety category, reaching max piety with 21 gods and winning with 21 gods. They also got the second most points for combos won, including the Te, Dj, and On species high scores, and won 13 Nemelex Choice combos.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/flugkiller.html">Flugkiller</a>: 3rd place overall with 99,209 points and 13 wins. He got a 0:25:20 realtime <a href="http://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20240129-191308.txt">MiFi</a> win (2nd overall), collected 19 tier 3 banners (2nd overall), won a <a href="http://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20240130-193816.txt">TrFi</a> in only 14,063 turns (3rd overall), got a 7 win streak, and won 6 Nemelex Choice combos.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/themeinteam.html">TheMeInTeam</a>: Longest win streak with 19 wins and highest in the winrate category (won 22 of 23 games played).</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/sapher.html">Sapher</a>: Highest scoring win at 91M points with an <a href="http://crawl.xtahua.com/crawl/morgue/Sapher/morgue-Sapher-20240124-173547.txt">OpEE</a>.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/enegeticocto.html">EnegeticOcto</a>: Fastest realtime win with a 0:22:48 <a href="http://crawl.xtahua.com/crawl/morgue/EnegeticOcto/morgue-EnegeticOcto-20240120-211134.txt">MiFi</a>.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/qw.html">qw</a>: Fastest realtime win by a <a href="https://github.com/crawl/qw">bot</a> with a 0:10:50 <a href="https://crawl.dcss.io/crawl/morgue/qw/morgue-qw-20240124-012829.txt">DjHu</a>. qw won 3 games this tournament, including first ever wins of Dj and Dg (with <a href="https://crawl.dcss.io/crawl/morgue/qw/morgue-qw-20240202-063259.txt">DgHu</a>). It also got 3 runes twice this tournament with SpHu.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/zran.html">zran</a>: Lowest turncount win with an <a href="https://crawl.kelbi.org/crawl/morgue/zran/morgue-zran-20240202-012159.txt">AtCj</a> winning in 10,274 turns.</li>
<li><a href="https://crawl.develz.org/tournament/0.31/players/bookman.html">bookman</a>: 23 Nemelex&#8217;s Choice wins, the most in the tournament.</li>
<li>The first three wins of the tournament were by FlugKiller (a <a href="https://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20240119-204641.txt">MiFi</a> in 0:46:25), <a href="https://crawl.develz.org/tournament/0.31/players/shadowrider38.html">ShadowRider38</a> (a <a href="https://cbro.berotato.org/morgue/shadowrider38/morgue-shadowrider38-20240119-204843.txt">MiFi</a> in 0:41:59), and <a href="https://crawl.develz.org/tournament/0.31/players/p0werm0de.html">p0werm0de</a> (a <a href="https://crawl.xtahua.com/crawl/morgue/p0werm0de/morgue-p0werm0de-20240119-205256.txt">MiFi</a> in 0:27:52; 3rd fastest realtime win overall!).</li>
<li>The last win of the tournament was by <a href="https://crawl.develz.org/tournament/0.31/players/sewnkin.html">sewnkin</a>, an <a href="http://crawl.akrasiac.org/rawdata/sewnkin/morgue-sewnkin-20240204-195946.txt">OnEE</a> that ascended 14 seconds before the end of tournament.</li>
</ul>
<p>The <a href="https://crawl.develz.org/tournament/0.31/teams.html">clan competition</a> was won by <a href="https://crawl.develz.org/tournament/0.31/clans/retired-ogres-bocce-club-enegeticocto.html">Retired Ogres Bocce Club</a> (captained by EnegeticOcto) with 132,494 points. They bested the 2nd place clan in the Combo High Scores (1st overall), Best High Score (1st overall), Fastest Realtime Win (1st overall), Most Pacific Win (1st overall), Fastest Realtime (1st overall), Low Turncount (1st overall), Banner Collection (tied for 1st overall), and Piety (1st overall) categories. In 2nd place was <a href="https://crawl.develz.org/tournament/0.31/clans/the-team-in-meee-cosplay-themeinteam.html">The TeAM in MeEE-Cosplay</a>(captained by TheMeInTeam) with 102,015 points, and first place in the Streak Length category, and 2nd place in the Piety and Nemelex Choice categories. Not far behind in 3rd place was <a href="https://crawl.develz.org/tournament/0.31/clans/harold-and-a-few-mara-go-to-wight-castle-wizard1ke.html">Harold and a Few Mara Go to Wight Castle</a> (captained by Wizard1ke) with 98,358 points, and 2nd place in the Combo High Scores, Best High Score, Low Turncount Win, and Most Pacific Win categories.</p>
<p>To close out, here are some overall tournament statistics (with the 0.30 numbers in brackets for comparison). 2,817 non-bot players started a game at some point during the tournament; of those 2,728 completed a game without quitting or leaving the dungeon sadly orbless. This definition of “player” (players who completed a non-boring game) is used for subsequent calculations and was used for the 0.30 statistics.</p>
<ul>
<li>Players: 2,728 (0.30: 2,708)</li>
<li>Total player time: 30,498 hrs (0.30: 30,426 hrs)</li>
<li>Average player time: 11.18 hrs (0.30: 11.24 hrs)</li>
<li>Games played: 71,851 (0.30: 69,571)</li>
<li>Players who got a rune: 861 (0.30: 926)</li>
<li>454 winners and 1367 wins (0.30: 454 winners and 1433 wins)</li>
<li>Win rate: 1.90% (0.30: 2.06%)</li>
<li>WebTiles win rate: 1.85% (0.30: 1.94%). Console win rate: 3.59% (0.30: 6.18%)</li>
</ul>
<p>Thanks to you all for playing! Many thanks as well to server admins, developers, and contributors who made the 0.31 release and tournament possible!</p>
<p>The 0.32 season has begun, and changes are already in motion. We have <a href="https://tavern.dcss.io/t/playable-experimentals-steel-elves-dual-wielding-beogh-yred-rework-now-with-mountain-dwarves/627">two online experimental branches</a> that implement a new Steel Elf species and redesigns the gods Beogh and Yredelemnul. The Legends Say that Watch this space for further information as the 0.32 trunk update posts begin to flow. Until next time, happy crawling!</p>


guid: https://crawl.develz.org/wordpress/?p=5450
title: CDI is Now Official and CKO is Going Offline
author: gammafunk
url: https://crawl.develz.org/wordpress/cdi-is-now-official-and-cko-is-going-offline
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Greetings Dungeon Crawlers! The official <a href="https://crawl.kelbi.org">crawl.kelbi.org</a> server, also known as <strong>CKO</strong>, is going offline tomorrow. Many thanks to floraline for generously hosting what was often our busiest crawl server since 2018! Additionally, floraline contributed several aspects to WebTiles configuration that continue to be used by other official servers to this day. Your CKO game records will of course continue to be available in Sequell and <a href="https://crawl.akrasiac.org/scoring/overview.html">CAO scoring</a>. We also have a new long-term home for your CKO morgue files. Those should become accessible at the new host in the next couple weeks, after which Sequell and CAO scoring will point to the new morgue URLs for all past CKO games.</p>
<p>Speaking of official servers, I&#8217;m upgrading the development server <a href="https://crawl.dcss.io/">crawl.dcss.io</a> aka <strong>CDI</strong> to become official! We&#8217;re using the cbro.berotato.org aka <strong>CBR2</strong> account db as the starting db for CDI&#8217;s accounts. Players who were already registered on CBR2 as of Friday March 30 can log into CDI with their CBR2 credentials, otherwise please register a new account on CDI. Users migrating from CKO should see very similar latency, since CDI is hosted in the same New York City data center where CKO was located.</p>
<p>Finally, I&#8217;ve set up a <a href="https://www.patreon.com/gammafunk">Patreon</a> to help cover hosting costs for CDI. Head over there for more details, and thanks for any support you can provide! Observant gizmo inventors might notice <a href="https://github.com/crawl/qw">qw</a> trying out the new Coglin species in trunk on CDI at some point soon. And who knows what kind of <a href="https://www.youtube.com/watch?v=rvJpiCjVrw0">memes</a> one might eventually find on our newest server? As always, happy crawling!</p>


guid: https://crawl.develz.org/wordpress/?p=5484
title: New Crawl Servers and a Possible Server Retirement
author: gammafunk
url: https://crawl.develz.org/wordpress/new-crawl-servers-and-a-possible-server-retirement
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Greetings, Dungeon Crawlers, as I once again bring you important server information!</p>
<p>We have a new server hosted in South Korea at <a href="https://crawl.nemelex.cards">crawl.nemelex.cards</a>, known officially as <strong>CNC</strong>. This server is owned by <strong>ASCIIPhilia</strong>, who has worked with us to make addition of a new server as painless as possible. This server uses a snapshot of the CPO user database from <em>June 19th, 2024</em> as its starting user database, which means users already registered on CPO at that time won&#8217;t have to re-register on CNC. Please see the the drop-down instructions at the top of the CNC lobby for detailed instructions on dealing with any account conflicts. On the CNC lobby page you&#8217;ll also find a link to the CNC chat Discord as well info on an improved version of the famous multi-purpose WebTiles extension module that ASCIPhilia developed for CWZ. Many thanks for ASCIIPhilia for generously hosting our newest official server!</p>
<p>Speaking of <strong>CWZ</strong>, unfortunately it suffered an SSD failure and is offline until further notice. Although the CWZ owner and admin, <strong>hong</strong>, is trying to have the server&#8217;s data recovered, it&#8217;s not clear if or when that will be completed. In any case, many thanks to hong for hosting CWZ, which has supported the South Korean DCSS community for over nine years! If CWZ does ever make a triumphant return, we&#8217;ll be sure to let you know. Regardless of this, CNC will continue to exist as its own independent server.</p>
<p>Next, <em>XTAHUA IS REJOICING</em> because <strong>CXC</strong> has a new owner and admin that has migrated all data to a new host! Many thanks to <strong>melvinkitnick</strong> for providing the new server long-term and for working with <strong>Namanix</strong>, the previous owner and admin, to migrate the data. This transition should be seamless for CXC users, since the user database, morgues, ttyrecs, etc were all copied over from the old host. And of course, many thanks to Namanix for providing a host for CXC for almost 5 years!</p>
<p>Unfortunately, the <strong>CPO</strong> server may be going offline in October unless a new owner can come forward and provide a replacement server. The CPO owner and admin, <strong>chequers</strong>, is able to provide the current server as-is as well as transfer the hostname to a new admin who&#8217;s willing provide the funding and has the necessary technical expertise. See the <a href="https://crawl.project357.org/">CPO Lobby Page</a> for further details. Many thanks to chequers for hosting CPO and supporting the Australian and Oceania DCSS community for 10 years!</p>
<p>Finally, a quick note that all games from CDI and CNC are now finally integrated into <a href="https://crawl.akrasiac.org/scoring/overview.html">CAO Scoring</a>. If you have games on any official server that aren&#8217;t showing up in the score pages, please let us know in #crawl-dev on Freenode or in the <a href="https://discord.gg/VEyx4Yk">Roguelikes Discord</a>. That&#8217;s all I have for now, so until next time, happy crawling!</p>


guid: https://crawl.develz.org/wordpress/?p=5512
title: 0.32 Release and Tournament
author: gammafunk
url: https://crawl.develz.org/wordpress/0-32-release-and-tournament
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>Hello crawlers! We&#8217;ve entered feature freeze for 0.32, in preparation for the impending release and tournament. The tournament will run from <strong>Friday August 30th 2024 at 20:00 UTC </strong>until <strong>Sunday September 15th 2024 at 20:00 UTC</strong>. An initial version of the official <a href="https://crawl.develz.org/tournament/0.32/">0.32 tournament page</a> is now available for bookmarking. We don&#8217;t expect many rule changes compared to 0.31, but any such changes will be noted at the bottom of that page before the tournament begins. Note that the pages will be in Test Mode and may have random test data in coming days, but any such data will be wiped before the tournament starts.</p>
<p>The<strong> 0.32 release</strong> will happen a day or two before August 30th. Also stay tuned for a forthcoming <strong>Trunk Update</strong> covering all the changes in 0.32. This post will mostly be a summary of the now up-to-date <a href="https://github.com/crawl/crawl/blob/master/crawl-ref/docs/changelog.txt">changelog</a>, so take a look there if you&#8217;re dying to know what spells Xom has taught rats to cast. Best of luck to you all in the tournament!</p>


guid: https://crawl.develz.org/wordpress/?p=5525
title: 0.32 “Gods and Makers”
author: gammafunk
url: https://crawl.develz.org/wordpress/0-32-gods-and-makers
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>We are pleased to announce the release of <strong>Dungeon Crawl: Stone Soup 0.32 “Gods and Makers”</strong>!</p>
<p>DCSS 0.32 features redesigns of four gods, two new playable species, eleven new spells, many new and revamped monsters, as well as new decorations throughout the Dungeon! For the complete list of these and the many other changes included in this version, see the release page <a href="https://github.com/crawl/crawl/releases/tag/0.32.0">here</a>.</p>
<p>Packages of DCSS 0.32 for Windows, OS X, and Linux are all available from the <a href="https://crawl.develz.org/download.htm">download page</a>. You can also <a href="https://crawl.develz.org/play.htm">play it online</a> on one of many servers across the world!</p>
<p>The tournament starts on <em>Friday August 30th, 8pm UTC</em> and runs through <em>Sunday September 15th, 8pm UTC</em>, with all online 0.32 games counting towards your score. See the <a href="https://crawl.develz.org/tournament/0.32/">tournament page</a> for more details, including how to join or setup a clan. Clans membership can be changed until Friday September 6th, 8pm UTC.</p>
<p>Many thanks to all those who have contributed to DCSS over the years. Thanks especially to the DCSS development team, whose contributions made 0.32 possible, and to the admins who run and maintain the official servers. A special thanks to the following community members who contributed to 0.32 and a warm welcome to the first-time contributors.</p>
<blockquote><p>910rd, Alex Jurkiewicz, Aliscans, Andrew O&#8217;Neill, bjiorn, caryoscelus, Christopher Partin, Chuck Sellick, CipHuK, DevJac, elliottbernstein, geekosaur, grumposus, Isaac Clancy, Lexi Hattaway, mainiacjoe, mgdelmonte, michaelhunter27, Monkooky, NormalPerson7, orjb1, Quinten Konyn, RypoFalem, Rytis Petronis, Sean Dewar, SentientSupper, Wizard Ike, yrdzrfxndfvh, Zhang Kai</p></blockquote>


guid: https://crawl.develz.org/wordpress/?p=5548
title: 0.32 Tournament Results
author: gammafunk
url: https://crawl.develz.org/wordpress/0-32-tournament-results
feedurl: https://crawl.develz.org/wordpress/feed
content_mime_type: text/html
content:
<p>The <a title="0.31 tournament" href="https://crawl.develz.org/tournament/0.32/overview.html">0.32 tournament</a> is over. On behalf of the Dev Team, many thanks to all the server admins, outside contributors, bug reporters, and to the many DCSS players who made the 0.32 release and tournament possible! For 16 days, players could compete for tournament points and banners by playing 0.32 games on the public servers. The only major rule change compared to the 0.31 tournament is that the Ziggurat category scoring was capped at 5 completed zigs. Zigs completed past 5 are still displayed for bragging rights, of course!</p>
<p>Congratulations to <a href="https://crawl.develz.org/tournament/0.32/players/flugkiller.html">Flugkiller</a> for taking first place in the <a href="https://crawl.develz.org/tournament/0.32/all-players-ranks.html">individual competition</a>, with 114,782 points and 27 wins! Flugkiller won the most Nemelex Choice combos (19), got all tier 3 banners, one of only two players to do this, and took 1st in the Piety category. He also got the second-fastest real-time win, a <a href="http://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20240902-005543.txt">MiFi</a> won in 00:24:58, and won the second-highest scoring game, an <a href="http://crawl.xtahua.com/crawl/morgue/Flugkiller/morgue-Flugkiller-20240904-122525.txt">AtCj</a> with 74M points.</p>
<p>Other big winners:</p>
<ul>
<li><a href="https://crawl.develz.org/tournament/0.32/players/themeinteam.html">TheMeInTeam</a>: 2nd place overall with 97,543 points and 22 wins. TheMeInTeam tied for the longest win streak of the tournament with 10 wins, and was 2nd in Nemelex Choice combos (18 wins) and Piety categories. He was also 2nd in the Combo High Scores category, with species high scores for Coglin (<a href="https://cbro.berotato.org/morgue/TheMeInTeam/morgue-TheMeInTeam-20240910-005211.txt">CoWn</a>) and Kobold (<a href="https://cbro.berotato.org/morgue/TheMeInTeam/morgue-TheMeInTeam-20240907-030045.txt">KoCa)</a>, the Hexslinger class high score (<a href="https://cbro.berotato.org/morgue/TheMeInTeam/morgue-TheMeInTeam-20240913-031856.txt">MDHs</a>), and high scores for 8 combos in total.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/berryknight.html">BerryKnight</a>: 3rd place overall with 93,883 points and 11 wins. They won the 2nd most pacific game of the tournament, a <a href="http://crawl.akrasiac.org/rawdata/BerryKnight/morgue-BerryKnight-20240905-054004.txt">MiFi</a> with just 1044 kills, and took 3rd in the Combo High Scores category with a <a href="https://cbro.berotato.org/morgue/BerryKnight/morgue-BerryKnight-20240915-084759.txt">DgSh</a> that took the Dg species and Sh class high scores and the <a href="https://cbro.berotato.org/morgue/BerryKnight/morgue-BerryKnight-20240909-193508.txt">Na</a> species high score.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/tallodds.html">TallOdds</a> (aka CarefulOdds): Tied with TheMeInTeam for the longest win streak with 10 wins.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/sapher.html">Sapher</a>: Highest scoring win at 98M points with an <a href="https://crawl.develz.org/tournament/0.32/players/sapher.html">FoEE</a> that also set the all-time Formicid species high score!</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/p0werm0de.html">p0werm0de</a>: Fastest realtime win with a 0:24:57 <a href="https://crawl.xtahua.com/crawl/morgue/p0werm0de/morgue-p0werm0de-20240901-210046.txt">MiFi</a>.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/acrobatsimulacrum.html">acrobatsimulacrum</a>: The only other player along with Flugkiller to collect all 24 tier 3 banners.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/repoman.html">RepoMan</a>: 1st place in the Combo High Scores category with 6 species and 6 class high scores. Legends say that RepoMan won all these games using Wu Jian and a certain <a href="http://crawl.chaosforge.org/Repo_Troll_Guide,_TrFi%5EWu_Jian">wiki guide</a>.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/zran.html">zran</a> (aka narz): Lowest turncount win, a <a href="https://cbro.berotato.org/morgue/zran/morgue-zran-20240915-033412.txt">GnEE</a> won in 11,445 turns.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/sentinel.html">sentinel</a>: Most pacific win, a <a href="https://crawl.xtahua.com/crawl/morgue/sentinel/morgue-sentinel-20240912-215423.txt">TrDe</a> with only 964 kills.</li>
<li><a href="https://crawl.develz.org/tournament/0.32/players/qw.html">qw</a> (bot): The third win of the tournament, a <a href="https://crawl.dcss.io/crawl/morgue/qw/morgue-qw-20240830-223350.txt">GrFi</a> of Okawaru won in only 8 attempts. A 12.5% winrate is very good for qw and represents a combination of good luck and recent improvements to its code.</li>
<li>The first three wins of the tournament were by <a href="https://crawl.develz.org/tournament/0.32/players/enegeticocto.html">EnegeticOcto</a> (a <a href="http://crawl.xtahua.com/crawl/morgue/EnegeticOcto/morgue-EnegeticOcto-20240830-205551.txt">MiFi</a> in 0:54:59), <a href="https://crawl.develz.org/tournament/0.32/players/tempest.html">tempest</a> (a <a href="https://crawl.dcss.io/crawl/morgue/tempest/morgue-tempest-20240830-221517.txt">MiBe</a> in 1:37:22), and qw (a GrFi in 1:50:46).</li>
</ul>
<p>The <a href="https://crawl.develz.org/tournament/0.32/teams.html">clan competition</a> was won by <a href="https://crawl.develz.org/tournament/0.32/clans/xoms-cocktail-party-enegeticocto.html">XOMS CoCKtail Party</a> (captained by EnegeticOcto) with 131,548 points. They bested the 2nd place clan in the Combo High Scores (1st overall), Best High Score (1st overall), Fastest Realtime Win (2nd overall), Lowest Turncount win (1st overall), Most Pacific win (1st overall), Banner Collection (tied for 1st overall), and Piety (1st overall) categories. In 2nd place was <a href="https://crawl.develz.org/tournament/0.32/clans/artifact-hats-only-cosplay-themeinteam.html">Artifact Hats Only &#8211; Cosplay</a> (captained by TheMeInTeam) with 117,174 points, 1st place in the Streak Length and Nemelex Choice categories, 2nd place in the Piety and Banner Collection categories, and 3rd place in the Fastest Realtime Win category. In 3rd place was <a href="https://crawl.develz.org/tournament/0.32/clans/teamsplat-mandevil.html">TeamSplat</a> (captained by <a href="https://crawl.develz.org/tournament/0.32/players/mandevil.html">Mandevil</a>) with 98,358 points and 2nd place in the Most Pacific Win category.</p>
<p>To close out, here are some overall tournament statistics (with the 0.31 numbers in brackets for comparison). 2,904 non-bot players started a game at some point during the tournament; of those 2,838 completed a game without quitting or leaving the dungeon sadly orbless. This definition of “player” (players who completed a non-boring game) is used for subsequent calculations and was used for the 0.31 statistics.</p>
<ul>
<li>Players: 2,838 (0.31: 2,728)</li>
<li>Total player time: 29,114 hrs (0.31: 30,498 hrs)</li>
<li>Average player time: 10.26 hrs (0.31: 11.18 hrs)</li>
<li>Games played: 76,121 (0.31: 71,851)</li>
<li>Players who got a rune: 871 (0.31: 861)</li>
<li>421 winners and 1319 wins (0.31: 454 winners and 1367 wins)</li>
<li>Win rate: 1.73% (0.31: 1.90%)</li>
<li>WebTiles win rate: 1.63% (0.31: 1.85%) / Console win rate: 5.43% (0.31: 3.59%)</li>
</ul>
<p>Thanks to you all for playing! Many thanks as well to server admins, developers, and contributors who made the 0.32 release and tournament possible! The 0.33 season has begun, and changes are already commit storming in. In the development branch, you can already try out several new player mutations, experience new and reworked Pandemonium monsters (both holy and demonic), and see some awesome new tile art! Also look for an important 0.32.1 bugfix release to made this week. That&#8217;s all for now, and until next time, happy crawling!</p>


guid: https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/
title: rsync, article 2: Surroundings
url: https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>This post is the second article in a series of blog posts about rsync, <a href="../2022-06-18-rsync-overview/">see the
Series Overview</a>.</p>
<p>Now that we know what to use rsync for, how can we best integrate rsync into
monitoring and alerting, and on which operating systems does it work?</p>
<h2 id="monitoring-and-alerting-for-rsync-jobs-using-prometheus">Monitoring and alerting for rsync jobs using Prometheus</h2>
<p>Once you have one or two important <code>rsync</code> jobs, it might make sense to alert
when your job has not completed as expected.</p>
<p>I’m using <a href="https://prometheus.io/">Prometheus</a> for all my monitoring and alerting.</p>
<p>Because Prometheus <em>pulls</em> metrics from its (typically always-running) targets,
we need an extra component: the <a href="https://prometheus.io/docs/practices/pushing/">Prometheus
Pushgateway</a>. The Pushgateway
stores metrics pushed by short-lived jobs like <code>rsync</code> transfers and makes them
available to subsequent Prometheus pulls.</p>
<p>To integrate <code>rsync</code> with the Prometheus Pushgateway, I wrote
<a href="https://github.com/stapelberg/rsyncprom"><code>rsyncprom</code></a>, a small tool that wraps
<code>rsync</code>, or parses rsync output supplied by you. Once <code>rsync</code> completes,
<code>rsyncprom</code> pushes the rsync exit code and parsed statistics about the transfer
to your Pushgateway.</p>
<h3 id="prometheus-server-side-setup">Prometheus server-side setup</h3>
<p>First, I set up the Prometheus Pushgateway (via Docker and systemd) on my
server.</p>
<p>Then, in my <code>prometheus.conf</code> file, I instruct Prometheus to pull data from my
Pushgateway:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># prometheus.conf</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">rule_files</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span>- backups.rules.yml<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">scrape_configs</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#60a0b0;font-style:italic"># […]</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span>- <span style="color:#062873;font-weight:bold">job_name</span>:<span style="color:#bbb"> </span>pushgateway<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">honor_labels</span>:<span style="color:#bbb"> </span><span style="color:#007020;font-weight:bold">true</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">static_configs</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span>- <span style="color:#062873;font-weight:bold">targets</span>:<span style="color:#bbb"> </span>[<span style="color:#4070a0">&#39;pushgateway:9091&#39;</span>]<span style="color:#bbb">
</span></span></span></code></pre></div><p>Finally, in <code>backups.rules.yml</code>, I configure an alert on the time series <code>rsync_exit_code</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># backups.rules.yml</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">groups</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span>- <span style="color:#062873;font-weight:bold">name</span>:<span style="color:#bbb"> </span>backups.rules<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">rules</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span>- <span style="color:#062873;font-weight:bold">alert</span>:<span style="color:#bbb"> </span>RsyncFailing<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">expr</span>:<span style="color:#bbb"> </span>rsync_exit_code{job=&#34;rsync&#34;} &gt; 0<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">for</span>:<span style="color:#bbb"> </span>1m<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">labels</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">      </span><span style="color:#062873;font-weight:bold">job</span>:<span style="color:#bbb"> </span>rsync<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">annotations</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">      </span><span style="color:#062873;font-weight:bold">description</span>:<span style="color:#bbb"> </span>rsync {{ $labels.instance }} is failing<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">      </span><span style="color:#062873;font-weight:bold">summary</span>:<span style="color:#bbb"> </span>rsync {{ $labels.instance }} is failing<span style="color:#bbb">
</span></span></span></code></pre></div><p>This alert will fire any time an rsync job monitored via <code>rsyncprom</code> exits with
a non-zero exit code.</p>
<h3 id="rsync-client-side-setup">rsync client-side setup</h3>
<p>On each machine that runs <code>rsync</code> jobs I want to monitor, I first install
<code>rsyncprom</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>go install github.com/stapelberg/rsyncprom/cmd/rsync-prom@latest
</span></span></code></pre></div><p>Then, I just wrap <code>rsync</code> transfers where it’s most convenient, for example in
my <a href="https://manpages.debian.org/crontab.5"><code>crontab(5)</code></a>
:</p>
<pre tabindex="0"><code class="language-crontab" data-lang="crontab"># crontab -e
9 9 * * * /home/michael/go/bin/rsync-prom --job=&#34;cron&#34; --instance=&#34;gphotos-sync@midna&#34; -- /home/michael/gphotos-sync/sync.sh
</code></pre><p>The same wrapper technique works in <a href="https://github.com/stapelberg/rsyncprom#setup-example-systemd">shell scripts or systemd service
files</a>.</p>
<p>You can also provide <code>rsync</code> output <a href="https://github.com/stapelberg/zkj-nas-tools/blob/02d46d718df60c413844d9218f6dd702ad94e5f1/dornroeschen/sshutil.go#L134-L139">from Go
code</a>
(this example runs <code>rsync</code> via SSH).</p>
<h3 id="monitoring-architecture">Monitoring architecture</h3>
<p>Here’s how the whole setup looks like architecturally:</p>
<p><img src="2022-05-29-rsync-prom-architecture.svg" alt="rsync-prom architecture"></p>
<p>The rsync scheduler runs on a Raspberry Pi running
<a href="https://gokrazy.org/">gokrazy</a>. The scheduler invokes the <code>rsync</code> job to back
up <em>websrv.zekjur.net</em> via SSH and sends the output to Prometheus, which is
running on a (different) server at an ISP.</p>
<h3 id="monitoring-dashboard">Monitoring dashboard</h3>
<p>The grafana dashboard looks like this in action:</p>















<a href="https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/2022-06-12-rsync-grafana-featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/2022-06-12-rsync-grafana-featured_hu49255cbd0611171ac9ea093d82565708_603964_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/2022-06-12-rsync-grafana-featured_hu49255cbd0611171ac9ea093d82565708_603964_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2022-07-02-rsync-surroundings/2022-06-12-rsync-grafana-featured_hu49255cbd0611171ac9ea093d82565708_603964_600x0_resize_q75_box.jpg"
  alt="rsync grafana dashboard" title="rsync grafana dashboard"
  width="600"
  height="469"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<ul>
<li>The top left table shows the most recent rsync exit code, green means 0 (success).</li>
<li>The top right graph shows rsync runtime (wall-clock time) over time. Long
runtime can have any number of bottlenecks as the reason: network connections,
storage devices, slow CPUs.</li>
<li>The bottom left graph shows rsync dataset size over time. This allows you to
quickly pinpoint transfers that are filling your disk up.</li>
<li>The bottom right graph shows transferred bytes per rsync over time. The higher
the value, the higher the amount of change in your data set between
synchronization runs.</li>
</ul>
<h2 id="rsync-operating-system-availability">rsync operating system availability</h2>
<p>Now that we have learnt about a couple of typical use-cases, where can you use
<code>rsync</code> to implement these use-cases? The answer is: in most environments, as
<code>rsync</code> is widely available on different Linux and BSD versions.</p>
<p>Macs come with <code>rsync</code> available by default (but it’s an old, patched version),
and OpenBSD comes with a BSD-licensed implementation called
<a href="https://www.openrsync.org/">openrsync</a> by default.</p>
<p>On Windows, you can use the <a href="https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux">Windows Subsystem for
Linux</a>.</p>
<table>
<thead>
<tr>
<th>Operating System</th>
<th>Implementation</th>
<th>Version</th>
</tr>
</thead>
<tbody>
<tr>
<td>FreeBSD 13.1 (ports)</td>
<td>tridge</td>
<td><a href="https://www.freebsd.org/cgi/man.cgi?query=rsync&amp;manpath=FreeBSD+13.1-RELEASE+and+Ports">3.2.3</a></td>
</tr>
<tr>
<td>OpenBSD 7.1</td>
<td>openrsync</td>
<td>(7.1)</td>
</tr>
<tr>
<td>OpenBSD 7.1 (ports)</td>
<td>tridge</td>
<td><a href="https://openports.se/net/rsync">3.2.4</a></td>
</tr>
<tr>
<td>NetBSD 9.2 (pkgsrc)</td>
<td>tridge</td>
<td><a href="https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/rsync/index.html">3.2.4</a></td>
</tr>
<tr>
<td>Linux</td>
<td>tridge</td>
<td><a href="https://repology.org/project/rsync/versions">repology</a></td>
</tr>
<tr>
<td>macOS</td>
<td>tridge</td>
<td><a href="https://www.unix.com/man-page/osx/1/rsync/">2.6.9</a></td>
</tr>
</tbody>
</table>
<h2 id="next-up">Next Up</h2>
<p>The third article in this series is <a href="../2022-07-02-rsync-how-does-it-work/">rsync, article 3: How does rsync
work?</a>. With rsync up and running, it’s
time to take a peek under the hood of rsync to better understand how it works.</p>


guid: https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/
title: rsync, article 3: How does rsync work?
url: https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>This post is the third article in a series of blog posts about rsync, <a href="../2022-06-18-rsync-overview/">see the
Series Overview</a>.</p>
<p>With rsync up and running, it’s time to take a peek under the hood of rsync to
better understand how it works.</p>
<h2 id="howdoesrsyncwork">How does rsync work?</h2>
<p>When talking about the rsync protocol, we need to distinguish between:</p>
<ul>
<li>protocol-level roles: “sender” and “receiver”</li>
<li>TCP roles: “client” and “server”</li>
</ul>
<p>All roles can be mixed and matched: both rsync clients (or servers!) can
either send or receive.</p>
<p>Now that you know the terminology, let’s take a high-level look at the rsync
protocol. We’ll look at protocol version 27, which is older but simpler, and
which is the most widely supported protocol version, implemented by <code>openrsync</code>
and other third-party implementations:</p>
<p><img src="2022-05-29-rsync-logical-view.svg" alt="rsync architecture"></p>
<p>The rsync protocol can be divided into two phases:</p>
<ol>
<li>
<p>In the first phase, the sender walks the local file tree to generate and send
the <em>file list</em> to the receiver. The file list must be transferred in full,
because both sides sort it by filename (later rsync protocol versions
eliminate this synchronous sorting step).</p>
</li>
<li>
<p>In the second phase, concurrently:</p>
<ul>
<li>The receiver compares and requests each file in the file list. The
receiver requests the full file when it didn’t exist on disk yet, or it
will send checksums for the rsync <em>hash search</em> algorithm when the file
already existed.</li>
<li>The receiver receives file data from the sender. The sender answers the
requests with just enough data to reconstruct the current file contents
based on what’s already on the receiver.</li>
</ul>
</li>
</ol>
<p>The architecture makes it easy to implement the second phase in 3 separate
processes, each of which sending to the network as fast as possible using heavy
pipelining. This results in utilizing the available hardware resources (I/O,
CPU, network) on sender and receiver to the fullest.</p>
<h3 id="observing-rsyncs-transfer-phases">Observing rsync’s transfer phases</h3>
<p>When starting an rsync transfer, looking at the resource usage of both
machines allows us to confirm our understanding of the rsync architecture, and
to pin-point any bottlenecks:</p>















<a href="https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/2022-05-29-rsync-exo1-backup.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/2022-05-29-rsync-exo1-backup_hua2f50278895cfbee4dc18c7ea60b6d4a_2093260_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/2022-05-29-rsync-exo1-backup_hua2f50278895cfbee4dc18c7ea60b6d4a_2093260_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/2022-05-29-rsync-exo1-backup_hua2f50278895cfbee4dc18c7ea60b6d4a_2093260_600x0_resize_q75_box.jpg"
  
  width="600"
  height="331"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<ol>
<li>phase: The rsync sender needs 17 seconds to walk the file system and send
the file list. The rsync receiver reads from the network and writes into RAM
during that time.
<ul>
<li>This phase is random I/O (querying file system metadata) for the sender.</li>
</ul>
</li>
<li>phase: Afterwards, the rsync sender reads from disk and sends to the
network. The rsync receiver receives from the network and writes to disk.
<ul>
<li>The receiver does roughly the same amount of random I/O as the sender did
in phase 1, as it needs to create directories and request missing files.</li>
<li>The sender does sequential disk reads and possibly checksum calculation, if
the file(s) existed on the receiver side.</li>
</ul>
</li>
</ol>
<p>(Again, the above was captured using rsync protocol version 27, later rsync
protocol versions don’t synchronize after completing phase 1, but instead
interleave the phases more.)</p>
<h3 id="rsync-hash-search">rsync hash search</h3>
<p>Up until now, we have described the rsync protocol at a high level. Let’s zoom
into the hash search step, which is what many people might associate with the
term “rsync algorithm”.</p>
<p>When a file exists on both sides, rsync sender and receiver, the receiver first
divides the file into blocks. The block size is a rounded square root of the
file’s length. The receiver then sends the checksums of all blocks to the
sender. In response, the sender finds matching blocks in the file and sends only
the data needed to reconstruct the file on the receiver side.</p>
<p>Specifically, the sender goes through <strong>each byte</strong> of the file and tries to
match existing receiver content. To make this less computationally expensive,
rsync combines two checksums.</p>
<p>rsync first calculates what it calls the
<a href="https://github.com/WayneD/rsync/blob/0ac7ebceef70417355f25daf9e2fd94e84c49749/checksum.c#L182-L203">“sum1”</a>,
or “fast signature”. This is a small checksum (two <code>uint16</code>) that can be
calculated with minimal effort for a rolling window over the file data. tridge
rsync comes with <a href="https://github.com/WayneD/rsync/blob/v3.2.4/simd-checksum-x86_64.cpp">SIMD
implementations</a>
to further speed this up where possible.</p>
<p>Only if the sum1 matches will
<a href="https://github.com/WayneD/rsync/blob/0ac7ebceef70417355f25daf9e2fd94e84c49749/checksum.c#L205-L300">“sum2”</a>
(or “strong signature”) be calculated, a 16-byte MD4 hash. Newer protocol
versions allow negotiating the hash algorithm and support the much faster xxhash
algorithms.</p>
<p>If sum2 matches, the block is considered equal on both sides.</p>
<p>Hence, the best case for rsync is when a file has either not changed at all, or
shares as many full blocks of content as possible with the old contents.</p>
<h2 id="changing-data-sets">Changing data sets</h2>
<p>Now that we know how rsync works on the file level, let’s take a step back to
the data set level.</p>
<p>The easiest situation is when you transfer a data set that is not currently
changing. But what happens when the data set changes while your rsync transfer
is running? Here are two examples.</p>
<p><a href="https://github.com/Debian/debiman">debiman</a>, the manpage generator powering
<a href="https://manpages.debian.org/">manpages.debian.org</a> is running on a Debian VM on
which an rsync job periodically transfers the static manpage archive to
different static web servers across the world. The rsync job and debiman are
not sequenced in any way. Instead, debiman is careful to only ever <a href="https://github.com/Debian/debiman/blob/79b8abe8573daaaf15e4cf40e85daf309e00d454/internal/write/atomically.go#L23">atomically
swap out
files</a>
in its output directory, or add new files before it swaps out an updated index.</p>
<p>The second example, the <a href="https://en.wikipedia.org/wiki/PostgreSQL">PostgreSQL</a>
database management system, is the opposite situation: instead of having full
control over how files are laid out, here I don’t have control over how files
are written (this generalizes to any situation where the model of only ever
replacing files is not feasible). The data files which my Postgres installation
keeps on disk are not great to synchronize using rsync: they are large and
frequently change. Instead, I now exempt them from my rsync transfer and use <a href="https://manpages.debian.org/pg_dump.1"><code>pg_dump(1)</code></a>
 to create a snapshot of my databases instead.</p>
<p>To confirm rsync’s behavior regarding changing data sets in detail, I <a href="#confirmation-hack">modified
rsync to ask for confirmation</a> between generating the file
list and transferring the files. Here’s what I found:</p>
<ul>
<li>If files are added after rsync has transferred the file list, the new files
will just not be part of the transfer.</li>
<li>If a file vanishes between generating the file list and transfering the file,
rsync exits with status code 24, which <a href="https://manpages.debian.org/bullseye/rsync/rsync.1.en.html#EXIT_VALUES">its manpage documents as “Partial
transfer due to vanished source
files”</a>. My
<code>rsyncprom</code> monitoring wrapper <a href="https://github.com/stapelberg/rsyncprom/commit/bbf82934093b938d054d460a7f9764234d30bc17">offers a flag to treat exit code 24 like exit
code
0</a>,
because depending on the data set, vanishing files are expected.</li>
<li>If a file’s contents change (no matter whether the file grows, shrinks, or is
modified in-place) between generating the file list and the actual file
transfer, that’s not a problem — rsync will transfer the file contents as it
reads them once the transfer starts. Note that this might be an inconsistent
view of the data, depending on the application.
<ul>
<li>Ideally, don’t ever modify files within a data set that is rsynced. Instead,
atomically move complete files into the data set.</li>
</ul>
</li>
</ul>
<p>Another way of phrasing the above is that data consistency is not something that
rsync can in any way guarantee. It’s up to you to either live with the
inconsistency (often a good-enough strategy!), or to add an extra step that
ensures the data set you feed to rsync is consistent.</p>
<h2 id="next-up">Next up</h2>
<p>The fourth article in this series is rsync, article 4: My own rsync
implementation (To be published.)</p>
<h2 id="confirmation-hack">Appendix A: rsync confirmation hack</h2>
<p>For verifying rsync’s behavior with regards to changing data sets, I checked
out the following version:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>% git clone https://github.com/WayneD/rsync/ rsync-changing-data-sets
</span></span><span style="display:flex;"><span>% <span style="color:#007020">cd</span> rsync-changing-data-sets
</span></span><span style="display:flex;"><span>% git checkout v3.2.4
</span></span><span style="display:flex;"><span>% ./configure
</span></span><span style="display:flex;"><span>% make
</span></span></code></pre></div><p>Then, I modified <code>flist.c</code> to add a confirmation step between sending the file
list and doing the actual file transfers:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-diff" data-lang="diff"><span style="display:flex;"><span><span style="color:#000080;font-weight:bold">diff --git i/flist.c w/flist.c
</span></span></span><span style="display:flex;"><span><span style="color:#000080;font-weight:bold">index 1ba306bc..98981f34 100644
</span></span></span><span style="display:flex;"><span><span style="color:#000080;font-weight:bold"></span><span style="color:#a00000">--- i/flist.c
</span></span></span><span style="display:flex;"><span><span style="color:#a00000"></span><span style="color:#00a000">+++ w/flist.c
</span></span></span><span style="display:flex;"><span><span style="color:#00a000"></span><span style="color:#800080;font-weight:bold">@@ -20,6 +20,8 @@
</span></span></span><span style="display:flex;"><span><span style="color:#800080;font-weight:bold"></span>  * with this program; if not, visit the http://fsf.org website.
</span></span><span style="display:flex;"><span>  */
</span></span><span style="display:flex;"><span> 
</span></span><span style="display:flex;"><span><span style="color:#00a000">+#include &lt;stdio.h&gt;
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+
</span></span></span><span style="display:flex;"><span><span style="color:#00a000"></span> #include &#34;rsync.h&#34;
</span></span><span style="display:flex;"><span> #include &#34;ifuncs.h&#34;
</span></span><span style="display:flex;"><span> #include &#34;rounding.h&#34;
</span></span><span style="display:flex;"><span><span style="color:#800080;font-weight:bold">@@ -2516,6 +2518,17 @@ struct file_list *send_file_list(int f, int argc, char *argv[])
</span></span></span><span style="display:flex;"><span><span style="color:#800080;font-weight:bold"></span> 	if (DEBUG_GTE(FLIST, 2))
</span></span><span style="display:flex;"><span> 		rprintf(FINFO, &#34;send_file_list done\n&#34;);
</span></span><span style="display:flex;"><span> 
</span></span><span style="display:flex;"><span><span style="color:#00a000">+	char *line = NULL;
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	size_t llen = 0;
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	ssize_t nread;
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	printf(&#34;file list sent. enter &#39;yes&#39; to continue: &#34;);
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	while ((nread = getline(&amp;line, &amp;llen, stdin)) != -1) {
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	  if (nread == strlen(&#34;yes\n&#34;) &amp;&amp; strcasecmp(line, &#34;yes\n&#34;) == 0) {
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	    break;
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	  }
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	  printf(&#34;enter &#39;yes&#39; to continue: &#34;);
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+	}
</span></span></span><span style="display:flex;"><span><span style="color:#00a000">+
</span></span></span><span style="display:flex;"><span><span style="color:#00a000"></span> 	if (inc_recurse) {
</span></span><span style="display:flex;"><span> 		send_dir_depth = 1;
</span></span><span style="display:flex;"><span> 		add_dirs_to_tree(-1, flist, stats.num_dirs);
</span></span></code></pre></div><p>My rsync invocation is:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>./rsync -av --debug<span style="color:#666">=</span>all4 --protocol<span style="color:#666">=</span><span style="color:#40a070">27</span> ~/i3/src /tmp/DEST/
</span></span></code></pre></div><p>It’s necessary to use an older protocol version to make rsync generate a full
file list before starting the transfer. Later protocol versions interleave these
parts of the protocol.</p>


guid: https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/
title: DIY out-of-band management: remote console server
url: https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>For the guest WiFi at an event that eventually fell through, we wanted to tunnel
all the traffic through my internet connection via my home router.</p>
<p>Because the event is located in another country, many hours of travel away,
there are a couple of scenarios where remote control of my home router can be a
life-saver. For example, should my home router crash, remotely turning power off
and on again gets the event back online.</p>
<p>But, power-cycling a machine is a pretty big hammer. For some cases, like
locking yourself out with a configuration mistake, a more precise tool like a
remote serial console might be nicer.</p>
<p>In this article, I’ll present two cheap and pragmatic DIY out-of-band management
solutions that I have experimented with in the last couple of weeks and wanted
to share:</p>
<ul>
<li><a href="#power-only">Variant 1: Remote Power Management only</a></li>
<li><a href="#remote-console">Variant 2: a full Remote Console Server</a></li>
</ul>
<p>You can easily start with the first variant and upgrade it into the second
variant later.</p>
<h2 id="power-only">Variant 1: Remote Power Management</h2>
<h3 id="architecture-diagram">Architecture Diagram</h3>
<p>Here is the architecture of the system at a glance. The right-hand side is the
existing router I want to control, the left-hand side shows the out of band
management system:</p>




<a href="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/2022-07-02-power-mgmt-architecture.svg"><img
  src="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/2022-07-02-power-mgmt-architecture.svg"
  
  style="

border: 1px solid #000;

margin-right: 1rem"
  
  loading="lazy"></a>


<p>Let’s go through the hardware components from top to bottom.</p>
<h3 id="hardware-4g-wifi-router-out-of-band-network">Hardware: 4G WiFi Router (Out Of Band Network)</h3>
<p>The easiest way to have another network connection for projects like this one is
the <a href="https://www.digitec.ch/en/s1/product/digitec-iot-test-sim-card-data-flat-30-days-unlimited-10-mbits-sim-card-11689214?supplier=406802">digitec iot
subscription</a>. They
offer various different options, and their cheapest one, a 0.4 Mbps flatrate for
4 CHF per month, is sufficient for our use-case.</p>
<p>A convenient way of making the digitec iot subscription available to other
devices is to use a mobile WiFi router such as the <a href="https://www.digitec.ch/en/s1/product/tp-link-m7350-routers-5615329?supplier=406802">TP-Link M7350 4G/LTE Mobile
Wi-Fi
router</a>
(68 CHF). You can power it via USB, and it has a built-in battery that will last
for a few hours.</p>















<a href="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0484.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0484_hu855bbe0a9c47ac557e5b09db74fa3b06_1361683_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0484_hu855bbe0a9c47ac557e5b09db74fa3b06_1361683_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0484_hu855bbe0a9c47ac557e5b09db74fa3b06_1361683_600x0_resize_q75_box.jpg"
  alt="TP-Link M7350 4G/LTE Mobile Wi-Fi router connected to digitec iot" title="TP-Link M7350 4G/LTE Mobile Wi-Fi router connected to digitec iot"
  width="600"
  height="450"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<p>By default, the device turns itself off after a while when it thinks it is
unused, which is undesired for us — if the smart plug drops out of the WiFi, we
don’t want the whole system to go offline. You can turn off this behavior in the
web interface under <code>Advanced → Power Saving → Power Saving Mode</code>.</p>
<h3 id="hardware-wifi-smart-plug">Hardware: WiFi Smart Plug</h3>
<p>With the out of band network connection established, all you need to remotely
toggle power is a smart plug such as the <a href="https://itead.cc/product/sonoff-s26-wifi-smart-plug/">Sonoff S26 WiFi Smart
Plug</a>.</p>
<p>The simplest setup is to connect the Smart Plug to the 4G router via WiFi, and
control it using Sonoff’s mobile app via Sonoff’s cloud.</p>
<h4 id="non-cloud-solution">Non-cloud solution</h4>
<p>Alternatively, if you want to avoid the Sonoff cloud, the device comes with a
“DIY mode”, but <a href="https://twitter.com/zekjur/status/1321949087160258562">the DIY mode wouldn’t work reliably for
me</a> when I tried
it. Instead, I flashed the <a href="https://tasmota.github.io/docs/">Open Source Tasmota
firmware</a> and connected it to a self-hosted
MQTT server via the internet.</p>
<p>Of course, now your self-hosted MQTT server is a single point of failure, but
perhaps you prefer that over the Sonoff cloud being a single point of failure.</p>
<h2 id="remote-console">Variant 2: Remote Console Server</h2>
<p>Turning power off and on remotely is a great start, but what if you need actual
remote access to a system? In my case, I’m using a <a href="https://en.wikipedia.org/wiki/Serial_port">serial
port</a> to see log messages and run a
shell on my router. This is also called a “serial console”, and any device that
allows accessing a serial console without sitting physically in front of the
serial port is called a “remote console server”.</p>
<p>Commercially available remote console servers typically offer lots of ports (up
to 48) and cost lots of money (many thousand dollars or equivalent), because
their target application is to be installed in a rack full of machines in a lab
or data center. A few years ago, I built
<a href="https://freetserv.github.io/">freetserv</a>, an open source, open hardware
solution for this problem.</p>
<p>For the use-case at hand, we only need a single serial console, so we’ll do it
with a Raspberry Pi.</p>
<h3 id="architecture-diagram-1">Architecture Diagram</h3>
<p>The architecture for this variant looks similar to the other variant, but adds
the <em>consrv</em> Raspberry Pi Zero 2 W and a USB-to-serial adapter:</p>




<a href="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/2022-06-19-consrv_architecture.svg"><img
  src="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/2022-06-19-consrv_architecture.svg"
  
  style="

border: 1px solid #000;

margin-right: 1rem"
  
  loading="lazy"></a>


<h3 id="hardware-raspberry-pi-zero-2-w">Hardware: Raspberry Pi Zero 2 W</h3>
<p>We’ll use a <a href="https://www.raspberrypi.com/products/raspberry-pi-zero-2-w/">Raspberry Pi Zero 2
W</a> as our console
server. While the device is a little slower than a Raspberry Pi 3 B, it is still
plenty fast enough for providing a serial console, and it only consumes 0.8W of
power (see <a href="https://gokrazy.org/platforms/">gokrazy → Supported platforms</a> for
a comparison):</p>















<a href="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0767_featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0767_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_860347_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0767_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_860347_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2022-08-27-out-of-band-remote-console/IMG_0767_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_860347_600x0_resize_q75_box.jpg"
  alt="Raspberry Pi Zero 2 W with USB hub, ethernet and serial" title="Raspberry Pi Zero 2 W with USB hub, ethernet and serial"
  width="600"
  height="600"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<p>If the Pi Zero 2 W is not available, you can try using any other <a href="https://gokrazy.org/platforms/">Raspberry Pi
supported by gokrazy</a>, or even an older Pi Zero
with the <a href="https://gokrazy.org/platforms/#community-supported-raspberry-pi-os-32-bit-kernelfirmware">community-supported Pi OS 32-bit
kernel</a>
(I didn’t test that).</p>
<p>Our Pi will have at least two tasks:</p>
<ol>
<li>With a USB-to-serial adapter, the Pi will provide a serial console.</li>
<li>The Pi will run <a href="https://tailscale.com/">Tailscale</a> mesh networking, which
will transparently use either the wired network or fail over to the Out Of
Band network. Tailscale also frees us from setting up port forwardings,
dynamic DNS or anything like that.</li>
<li>Optionally, the Pi can run a local MQTT server if you want to avoid the
Sonoff cloud.</li>
</ol>
<h3 id="hardware-usb-to-serial-adapter">Hardware: USB-to-serial adapter</h3>
<p>You can use any USB-to-serial adapter supported by Linux. Personally, I like the
<a href="https://www.adafruit.com/product/2264">Adafruit FT232H adapter</a>, which I like
to <a href="https://twitter.com/zekjur/status/1256879027266224128">re-program with FTDI’s FT_Prog so that it has a unique serial
number</a>.</p>
<p>In my router, I plugged in an <a href="https://twitter.com/zekjur/status/1443461234930634755">Longshine LCS-6321M serial PCIe
card</a> to add a serial
port. Before you ask: no, <a href="https://twitter.com/zekjur/status/1439612800561819649">using USB serial consoles for the kernel
console</a> does not cut it.</p>
<h3 id="hardware-usb-ethernet-adapter">Hardware: USB ethernet adapter</h3>
<p>Because we not only want this Raspberry Pi to be available via the Out Of Band
network (via WiFi), but also on the regular home network, we need a USB ethernet
adapter.</p>
<p>Originally I was going to use the Waveshare ETH-USB-HUB-BOX: Ethernet / USB HUB
BOX for Raspberry Pi Zero Series, but it <a href="https://twitter.com/zekjur/status/1538582804224782337">turned out to be
unreliable</a>.</p>
<p>Instead, I’m now connecting a USB hub (as the Pi Zero 2 W has only one USB
port), a <a href="https://www.linksys.com/support-product?sku=USB3GIG">Linksys USB3GIG</a>
network adapter I had lying around, and my USB-to-serial adapter.</p>
<h3 id="gokrazy-setup">gokrazy setup</h3>
<p>Just like in the <a href="https://gokrazy.org/quickstart/">gokrazy quickstart</a>, we’re
going to create a directory for this gokrazy instance:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#bb60d5">INSTANCE</span><span style="color:#666">=</span>gokrazy/consrv
</span></span><span style="display:flex;"><span>mkdir -p ~/<span style="color:#70a0d0">${</span><span style="color:#bb60d5">INSTANCE</span>?<span style="color:#70a0d0">}</span>
</span></span><span style="display:flex;"><span><span style="color:#007020">cd</span> ~/<span style="color:#70a0d0">${</span><span style="color:#bb60d5">INSTANCE</span>?<span style="color:#70a0d0">}</span>
</span></span><span style="display:flex;"><span>go mod init consrv
</span></span></code></pre></div><p>You could now directly run <code>gokr-packer</code>, but personally, I like putting the
<code>gokr-packer</code> command into a
<a href="https://en.wikipedia.org/wiki/Make_(software)#Makefile"><code>Makefile</code></a> right away:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-makefile" data-lang="makefile"><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># The consrv hostname resolves to the device’s Tailscale IP address,
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># once Tailscale is set up.
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span><span style="color:#bb60d5">PACKER</span> <span style="color:#666">:=</span> gokr-packer -hostname<span style="color:#666">=</span>consrv
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#bb60d5">PKGS</span> <span style="color:#666">:=</span> <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/breakglass <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/timestamps <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/serial-busybox <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/stat/cmd/gokr-webstat <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/stat/cmd/gokr-stat <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/mkfs <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/gokrazy/wifi <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	tailscale.com/cmd/tailscaled <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	tailscale.com/cmd/tailscale <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>	github.com/mdlayher/consrv/cmd/consrv
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#06287e">all</span><span style="color:#666">:</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#06287e">.PHONY</span><span style="color:#666">:</span> update overwrite
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#06287e">update</span><span style="color:#666">:</span>
</span></span><span style="display:flex;"><span>	<span style="color:#70a0d0">${</span><span style="color:#bb60d5">PACKER</span><span style="color:#70a0d0">}</span> -update<span style="color:#666">=</span>yes <span style="color:#70a0d0">${</span><span style="color:#bb60d5">PKGS</span><span style="color:#70a0d0">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#06287e">overwrite</span><span style="color:#666">:</span>
</span></span><span style="display:flex;"><span>	<span style="color:#70a0d0">${</span><span style="color:#bb60d5">PACKER</span><span style="color:#70a0d0">}</span> -overwrite<span style="color:#666">=</span>/dev/sdx <span style="color:#70a0d0">${</span><span style="color:#bb60d5">PKGS</span><span style="color:#70a0d0">}</span>
</span></span></code></pre></div><p>For the initial install, plug the SD card into your computer, put its device
name into the <code>overwrite</code> target, and run <code>make overwrite</code>.</p>
<p>For subsequent changes, you can use <code>make update</code>.</p>
<h3 id="tailscale">Tailscale</h3>
<p>Tailscale is a peer-to-peer mesh VPN, meaning we can use it to connect to our
<code>consrv</code> Raspberry Pi from anywhere in the world, without having to set up port
forwardings, dynamic DNS, or similar.</p>
<p>As an added bonus, Tailscale also transparently fails over between connections,
so while the fast ethernet/fiber connection works, Tailscale uses that,
otherwise it uses the Out Of Band network.</p>
<p>Follow <a href="https://gokrazy.org/packages/tailscale/">the gokrazy guide on Tailscale</a>
to include the device in your Tailscale mesh VPN.</p>
<h3 id="wifi-internet-connection-and-dual-homing">WiFi internet connection and dual homing</h3>
<p>Setup WiFi:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>mkdir -p extrafiles/github.com/gokrazy/wifi/etc
</span></span><span style="display:flex;"><span>cat <span style="color:#4070a0">&#39;{&#34;ssid&#34;: &#34;oob&#34;, &#34;psk&#34;: &#34;secret&#34;}&#39;</span> <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  &gt; extrafiles/github.com/gokrazy/wifi/etc/wifi.json
</span></span></code></pre></div><p><code>consrv</code> should use the Out Of Band mobile uplink to reach the internet. At the
same time, it should still be usable from my home network, too, to make gokrazy
updates go quickly.</p>
<p>We accomplish this using route priorities.</p>
<p>I arranged for the WiFi interface to have higher route priority (5) than the
ethernet interface (typically 1, but 11 in our setup thanks to the
<code>-extra_route_priority=10</code> flag):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>mkdir -p flags/github.com/gokrazy/gokrazy/cmd/dhcp
</span></span><span style="display:flex;"><span><span style="color:#007020">echo</span> <span style="color:#4070a0">&#39;-extra_route_priority=10&#39;</span> <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  &gt; flags/github.com/gokrazy/gokrazy/cmd/dhcp/flags.txt
</span></span><span style="display:flex;"><span>make update
</span></span></code></pre></div><p>Now, <code>tailscale netcheck</code> shows an IPv4 address belonging to Sunrise, the mobile
network provider behind the digitec iot subscription.</p>
<h3 id="the-consrv-console-server">The consrv Console Server</h3>
<p><a href="https://github.com/mdlayher/consrv"><code>consrv</code></a> is an SSH serial console server
written in Go that Matt Layher and I developed. If you’re curious, you can watch
the two of us creating it in this twitch stream recording:</p>

<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
  <iframe src="https://www.youtube-nocookie.com/embed/1g46ei9aBH0" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe>
</div>

<p>The installation of <code>consrv</code> consists of two steps.</p>
<p>Step 1 is done: we already included <code>consrv</code> in the <code>Makefile</code> earlier in
<a href="#gokrazy-setup">gokrazy setup</a>.</p>
<p>So, we only need to configure the desired serial ports in <code>consrv.toml</code> (in
<a href="https://gokrazy.org/userguide/package-config/#extrafiles">gokrazy extrafiles</a>):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>mkdir -p extrafiles/github.com/mdlayher/consrv/cmd/consrv/etc/consrv
</span></span><span style="display:flex;"><span>cat &gt; extrafiles/github.com/mdlayher/consrv/cmd/consrv/etc/consrv/consrv.toml <span style="color:#4070a0">&lt;&lt;&#39;EOT&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">[server]
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">address = &#34;:2222&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">[[devices]]
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">serial = &#34;01716A92&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">name = &#34;router7&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">baud = 115200
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">logtostdout = true
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">[[identities]]
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">name = &#34;michael&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">public_key = &#34;ssh-ed25519 AAAAC3… michael@midna&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0">EOT</span>
</span></span></code></pre></div><p>Run <code>make update</code> to deploy the configuration to your device.</p>
<p>If everything is set up correctly, we can now start a serial console session via
SSH:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>midna% ssh -p 2222 router7@consrv.lan
</span></span><span style="display:flex;"><span>Warning: Permanently added &#39;[consrv.lan]:2222&#39; (ED25519) to the list of known hosts.
</span></span><span style="display:flex;"><span>consrv&gt; opened serial connection &#34;router7&#34;: path: &#34;/dev/ttyUSB0&#34;, serial: &#34;01716A92&#34;, baud: 115200
</span></span><span style="display:flex;"><span>2022/06/19 20:50:47 dns.go:175: probe results: [{upstream: [2001:4860:4860::8888]:53, rtt: 999.665µs} {upstream: [2001:4860:4860::8844]:53, rtt: 2.041079ms} {upstream: 8.8.8.8:53, rtt: 2.073279ms} {upstream: 8.8.4.4:53, rtt: 16.200959ms}]
</span></span><span style="display:flex;"><span>[…]
</span></span></code></pre></div><p>I’m using the <code>logtostdout</code> option to make <code>consrv</code> continuously read the serial
port and send it to <code>stdout</code>, which gokrazy in turn <a href="https://gokrazy.org/userguide/remotesyslog/">sends via remote
syslog</a> to the <a href="https://github.com/gokrazy/syslogd">gokrazy syslog
daemon</a>, running on another machine. You
could also run it on the same machine if you want to log to file.</p>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">There is an <a href="https://github.com/mdlayher/consrv/issues/3">open issue in
<code>consrv</code></a> regarding the failure
mode when a serial adapter disappears. Currently, <code>consrv</code> hangs until you try
to send something, then must be restarted. A workaround is available in the
GitHub issue.</div>
  </div>
</aside>

<h3 id="controlling-tasmota-from-breakglass">Controlling Tasmota from breakglass</h3>
<p>You can use <a href="https://github.com/gokrazy/breakglass"><code>breakglass</code></a> to
interactively log into your gokrazy installation.</p>
<p>If you flashed your Smart Plug with Tasmota, you can easily turn power on from a
breakglass shell by directly calling Tasmota’s HTTP API with <code>curl</code>:</p>
<pre tabindex="0"><code>% breakglass consrv
consrv# curl -v -X POST --data &#39;cmnd=power on&#39; http://tasmota_68462f-1583/cm
</code></pre><p>The original Sonoff firmware offers a DIY mode which should also offer an HTTP
API, but the <a href="https://twitter.com/zekjur/status/1321949087160258562">DIY mode did not work in my
tests</a>. Hence, I’m only
describing how to do it with Tasmota.</p>
<h3 id="optional-local-mqtt-server">Optional: Local MQTT Server</h3>
<p>Personally, I like having the Smart Plug available both on the local network
(via Tasmota’s HTTP API) and via the internet with an external MQTT server. That
way, even if either option fails, I still have a way to toggle power remotely.</p>
<p>But, maybe you want to obtain usage stats by listening to MQTT or similar, and
you don’t want to use an extra server for this. In that situation, you can
easily run a local MQTT server on your Pi.</p>
<p>In the gokrazy <code>Makefile</code>, add
<a href="https://github.com/fhmq/hmq"><code>github.com/fhmq/hmq</code></a> to the list of packages to
install, and configure Tasmota to connect to <code>consrv</code> on port 1883.</p>
<p>To check that everything is working, use <code>mosquitto_sub</code> from another machine:</p>
<pre tabindex="0"><code>midna% mosquitto_sub --verbose -h consrv.monkey-turtle.ts.net -t &#39;#&#39;
</code></pre><h2 id="conclusion">Conclusion</h2>
<p>digitec’s IOT mobile internet subscription makes remote power management
delightfully easy with a smart plug and 4G WiFi router, and affordable
enough. The subscription is flexible enough that you can decide to only book it
while you’re traveling.</p>
<p>We can elevate the whole setup in functionality (but also complexity) by
combining Tailscale, consrv and gokrazy, running on a Raspberry Pi Zero 2 W, and
connecting a USB-to-serial adapter.</p>
<p>If you need more features than that, check out the next step on the feature and
complexity ladder: <a href="https://pikvm.org/">PiKVM</a> or
<a href="https://tinypilotkvm.com/">TinyPilot</a>. See also <a href="https://www.jeffgeerling.com/blog/2021/raspberry-pi-kvms-compared-tinypilot-and-pi-kvm-v3">this comparison by Jeff
Geerling</a>.</p>
<h2 id="appendix-a-unstable-apple-usb-ethernet-adapter">Appendix A: Unstable Apple USB ethernet adapter</h2>
<p>The first USB ethernet adapter I tried was the <a href="https://www.artcomputer.ch/b2c_en/apple-usb-ethernet-adapter-a00004961/">Apple USB Ethernet
Adapter</a>.</p>
<p>Unfortunately, after a few days of uptime, I experienced the following kernel
driver crash (with the <code>asix</code> Linux driver), and the link remained down until I
rebooted.</p>
<p>I then switched to a <a href="https://www.linksys.com/support-product?sku=USB3GIG">Linksys
USB3GIG</a> network adapter
(supported by the <code>r8152</code> Linux driver) and did not see any problems with that
so far.</p>
<details>
<summary>kernel crash message (in dmesg)</summary>
<pre tabindex="0"><code>dwc2 3f980000.usb: dwc2_hc_chhltd_intr_dma: Channel 5 - ChHltd set, but reason is unknown
dwc2 3f980000.usb: hcint 0x00000002, intsts 0x04600009
dwc2 3f980000.usb: dwc2_update_urb_state_abn(): trimming xfer length
asix 1-1.4:1.0 eth0: Failed to read reg index 0x0000: -71
------------[ cut here ]------------
WARNING: CPU: 1 PID: 7588 at drivers/net/phy/phy.c:942 phy_error+0x10/0x58
Modules linked in: brcmfmac brcmutil
CPU: 1 PID: 7588 Comm: kworker/u8:2 Not tainted 5.18.3 #1
Hardware name: Raspberry Pi Zero 2 W Rev 1.0 (DT)
Workqueue: events_power_efficient phy_state_machine
pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : phy_error+0x10/0x58
lr : phy_state_machine+0x258/0x2b0
sp : ffff800009fe3d40
x29: ffff800009fe3d40 x28: 0000000000000000 x27: ffff6c7ac300c078
x26: ffff6c7ac300c000 x25: ffff6c7ac4390000 x24: 00000000ffffffb9
x23: 0000000000000004 x22: ffff6c7ac4019cd8 x21: ffff6c7ac4019800
x20: ffffce5c97f6f000 x19: ffff6c7ac4019800 x18: 0000000000000010
x17: 0000000400000000 x16: 0000000000000000 x15: 0000000000001007
x14: ffff800009fe3810 x13: 00000000ffffffea x12: 00000000fffff007
x11: fffffffffffe0290 x10: fffffffffffe0240 x9 : ffffce5c988e1018
x8 : c0000000fffff007 x7 : 00000000000000a8 x6 : ffffce5c98889280
x5 : 0000000000000268 x4 : ffff6c7acf392b80 x3 : ffff6c7ac4019cd8
x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff6c7ac4019800
Call trace:
 phy_error+0x10/0x58
 phy_state_machine+0x258/0x2b0
 process_one_work+0x1e4/0x348
 worker_thread+0x48/0x418
 kthread+0xf4/0x110
 ret_from_fork+0x10/0x20
---[ end trace 0000000000000000 ]---
asix 1-1.4:1.0 eth0: Link is Down
</code></pre></details>


guid: https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/
title: DIY out-of-band management: remote power button
url: https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>I was pleasantly surprised by how easy it was to make it possible to push a PC’s
power button remotely via MQTT by wiring up an ESP32 microcontroller, a MOSFET,
a resistor, and a few jumper wires.</p>
<p>While a commercial solution like IPMI offers many more features like remote
serial, or remote image mounting, this DIY solution feels really magical, and
has great price performance if all you need is power management.</p>















<a href="https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/IMG_1085_featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/IMG_1085_featured_huf7f7db33074cb268ff172c7a33b2b9a7_1568716_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/IMG_1085_featured_huf7f7db33074cb268ff172c7a33b2b9a7_1568716_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2022-10-09-remote-power-button/IMG_1085_featured_huf7f7db33074cb268ff172c7a33b2b9a7_1568716_600x0_resize_q75_box.jpg"
  alt="The inside of a PC case, where an ESP32 micro controller on an Adafruit Perma-Proto bread board is mounted inside the case and wired up to the mainboard with jumper wires for remote power control" title="The inside of a PC case, where an ESP32 micro controller on an Adafruit Perma-Proto bread board is mounted inside the case and wired up to the mainboard with jumper wires for remote power control"
  width="600"
  height="450"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<h2 id="motivation">Motivation</h2>
<p>To save power, I want to shut down my <a href="/posts/2019-10-23-nas/">network storage PC</a> when it isn’t currently needed.</p>
<p>For this plan to work out, my daily backup automation needs to be able to turn on the network storage PC, and power it back off when done.</p>
<p>Usually, I implement that via <a href="https://en.wikipedia.org/wiki/Wake-on-LAN">Wake On LAN
(WOL)</a>. But, for this particular
machine, I don’t have an ethernet network link, I only <a href="/posts/2020-08-09-fiber-link-home-network/">have a fiber
link</a>. Unfortunately, it seems like
none of the 3 different 10 Gbit/s network cards I tested has functioning Wake On
LAN, and when I asked on Twitter, none of my followers had ever seen functioning
WOL on any 10 Gbit/s card. I suppose it’s not a priority for the typical target
audience of these network cards, which go into always-on servers.</p>
<p>I didn’t want to run an extra 10 Gbit/s switch just for WOL over an ethernet
connection, because switches like the MikroTik CRS305-1G-4S+IN consume at least
10W. As the network storage PC only consumes about 20W overall, I wanted a more
power-efficient option.</p>
<h2 id="hardware-and-wiring">Hardware and Wiring</h2>
<p>The core of this DIY remote power button is a WiFi-enabled micro controller such
as the ESP32. To power the micro controller, I use the 5V standby power on the
mainboard’s USB 2.0 pin headers, which is also available when the PC is turned
off and only the power supply (PSU) is turned on. A micro controller with an
on-board 5V voltage regulator is convenient for this.</p>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">I verified the 5V standby power with a multimeter in DC power measurement
mode. Some embedded machines don’t have always-on 5V standby power, even if they
use an ATX power supply!</div>
  </div>
</aside>

<p>Aside from the micro controller, we also need a transistor or logic-level MOSFET
to simulate a push of the power button, and a resistor to control the
transistor. An opto coupler is not needed, since the ESP32 is powered from the
mainboard, not from a separate power supply.</p>
<p>The mainboard’s front panel header contains a <code>POWERBTN#</code> signal (3.3V), and a
<code>GND</code> signal. When connecting a typical PC case power button to the header, you
don’t need to pay attention to the polarity. This is because the power button
just physically connects the two signals.</p>
<p>In our case, the polarity matters, because we need the 3.3V on the transistor’s
drain pin, otherwise we won’t be able to control the transistor via its base
pin. The <code>POWERBTN#</code> 3.3V signal is typically labeled <code>+</code> on the mainboard (or
in the manual), whereas <code>GND</code> is labeled <code>-</code>. If you are unsure, double-check
the voltage using a multimeter.</p>
<h2 id="bill-of-materials">Bill of Materials</h2>
<ul>
<li>WiFi-enabled microcontroller with 5V power input, e.g. the <a href="https://docs.platformio.org/en/latest/boards/espressif32/pico32.html#board-espressif32-pico32">Espressif ESP32
Pico
Kit</a></li>
<li>transistor or logic-level MOSFET for working with 3.3V, e.g. <a href="https://www.digikey.com/en/products/detail/onsemi/2N7000/244278">2N7000
(→digikey)</a></li>
<li>1K resistor for controlling the transistor,
e.g. <a href="https://www.digikey.com/en/products/detail/stackpole-electronics-inc/CF14JT1K00/1741314">CF14JT1K00</a></li>
<li>a bread board and/or case for mounting, e.g. <a href="https://www.adafruit.com/product/571">Adafruit
Perma-Proto</a>.</li>
</ul>
<h2 id="schematic">Schematic</h2>
<p><a href="2022-10-08-remote-power-button.svg"><img src="2022-10-08-remote-power-button.svg" width="100%"></a></p>
<h2 id="software-esphome">Software: ESPHome</h2>
<p>I wanted a quick solution (with ideally no custom firmware development) and was
already familiar with <a href="https://esphome.io/">ESPHome</a>, which turns out to very
easily implement the functionality I wanted :)</p>
<p>In addition to a standard ESPHome configuration, I have added the following
lines to make the GPIO pin available through MQTT, and make it a momentary
switch instead of a toggle switch, so that it briefly presses the power button
and doesn’t hold the power button:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#062873;font-weight:bold">switch</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span>- <span style="color:#062873;font-weight:bold">platform</span>:<span style="color:#bbb"> </span>gpio<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">pin</span>:<span style="color:#bbb"> </span><span style="color:#40a070">25</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">id</span>:<span style="color:#bbb"> </span>powerbtn<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;powerbtn&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">restore_mode</span>:<span style="color:#bbb"> </span>ALWAYS_OFF<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">on_turn_on</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span>- <span style="color:#062873;font-weight:bold">delay</span>:<span style="color:#bbb"> </span>500ms<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span>- <span style="color:#062873;font-weight:bold">switch.turn_off</span>:<span style="color:#bbb"> </span>powerbtn<span style="color:#bbb">
</span></span></span></code></pre></div><p>I have elided the full configuration for brevity, but you can click here to see it:</p>
<details>
<summary>full ESPHome YAML configuration</summary>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#062873;font-weight:bold">esphome</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">name</span>:<span style="color:#bbb"> </span>poweresp<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">esp32</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">board</span>:<span style="color:#bbb"> </span>pico32<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">framework</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">type</span>:<span style="color:#bbb"> </span>arduino<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#60a0b0;font-style:italic"># Enable logging</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">logger</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">mqtt</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">broker</span>:<span style="color:#bbb"> </span><span style="color:#40a070">10.0.0.54</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">ota</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">password</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">wifi</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">ssid</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;essid&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">password</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;secret&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#60a0b0;font-style:italic"># Enable fallback hotspot (captive portal) in case wifi connection fails</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span><span style="color:#062873;font-weight:bold">ap</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">ssid</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;Poweresp Fallback Hotspot&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">password</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;secret2&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">captive_portal</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb"></span><span style="color:#062873;font-weight:bold">switch</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">  </span>- <span style="color:#062873;font-weight:bold">platform</span>:<span style="color:#bbb"> </span>gpio<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">pin</span>:<span style="color:#bbb"> </span><span style="color:#40a070">25</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">id</span>:<span style="color:#bbb"> </span>powerbtn<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#4070a0">&#34;powerbtn&#34;</span><span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">restore_mode</span>:<span style="color:#bbb"> </span>ALWAYS_OFF<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span><span style="color:#062873;font-weight:bold">on_turn_on</span>:<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span>- <span style="color:#062873;font-weight:bold">delay</span>:<span style="color:#bbb"> </span>500ms<span style="color:#bbb">
</span></span></span><span style="display:flex;"><span><span style="color:#bbb">    </span>- <span style="color:#062873;font-weight:bold">switch.turn_off</span>:<span style="color:#bbb"> </span>powerbtn<span style="color:#bbb">
</span></span></span></code></pre></div></details>
<p>For the first flash, I used:</p>
<pre tabindex="0"><code>docker run --rm \
  -v &#34;${PWD}&#34;:/config \
  --device=/dev/ttyUSB0 \
  -it \
  esphome/esphome \
    run poweresp.yaml
</code></pre><p>To update over the network after making changes (serial connection no longer needed), I used:</p>
<pre tabindex="0"><code>docker run --rm \
  -v &#34;${PWD}&#34;:/config \
  -it \
  esphome/esphome \
    run poweresp.yaml
</code></pre><p>In case you want to learn more about the relevant ESPHome concepts, here are a
few pointers:</p>
<ul>
<li><a href="https://esphome.io/components/wifi.html">https://esphome.io/components/wifi.html</a> might need to set <code>use_address</code></li>
<li><a href="https://esphome.io/components/switch/index.html">https://esphome.io/components/switch/index.html</a>
<ul>
<li>and <a href="https://esphome.io/components/switch/gpio.html">https://esphome.io/components/switch/gpio.html</a></li>
</ul>
</li>
<li><a href="https://esphome.io/components/mqtt.html">https://esphome.io/components/mqtt.html</a></li>
</ul>
<h2 id="integration-into-automation">Integration into automation</h2>
<p>To push the power button remotely from Go, I’m using the following code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">func</span> <span style="color:#06287e">pushMainboardPower</span>(mqttBroker, clientID <span style="color:#902000">string</span>) <span style="color:#902000">error</span> {
</span></span><span style="display:flex;"><span>	opts <span style="color:#666">:=</span> mqtt.<span style="color:#06287e">NewClientOptions</span>().<span style="color:#06287e">AddBroker</span>(mqttBroker)
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">if</span> hostname, err <span style="color:#666">:=</span> os.<span style="color:#06287e">Hostname</span>(); err <span style="color:#666">==</span> <span style="color:#007020;font-weight:bold">nil</span> {
</span></span><span style="display:flex;"><span>		clientID <span style="color:#666">+=</span> <span style="color:#4070a0">&#34;@&#34;</span> <span style="color:#666">+</span> hostname
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>	opts.<span style="color:#06287e">SetClientID</span>(clientID)
</span></span><span style="display:flex;"><span>	opts.<span style="color:#06287e">SetConnectRetry</span>(<span style="color:#007020;font-weight:bold">true</span>)
</span></span><span style="display:flex;"><span>	mqttClient <span style="color:#666">:=</span> mqtt.<span style="color:#06287e">NewClient</span>(opts)
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">if</span> token <span style="color:#666">:=</span> mqttClient.<span style="color:#06287e">Connect</span>(); token.<span style="color:#06287e">Wait</span>() <span style="color:#666">&amp;&amp;</span> token.<span style="color:#06287e">Error</span>() <span style="color:#666">!=</span> <span style="color:#007020;font-weight:bold">nil</span> {
</span></span><span style="display:flex;"><span>		<span style="color:#007020;font-weight:bold">return</span> fmt.<span style="color:#06287e">Errorf</span>(<span style="color:#4070a0">&#34;connecting to MQTT: %v&#34;</span>, token.<span style="color:#06287e">Error</span>())
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">const</span> topic = <span style="color:#4070a0">&#34;poweresp/switch/powerbtn/command&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">const</span> qos = <span style="color:#40a070">0</span> <span style="color:#60a0b0;font-style:italic">// at most once (no re-transmissions)
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>	<span style="color:#007020;font-weight:bold">const</span> retained = <span style="color:#007020;font-weight:bold">false</span>
</span></span><span style="display:flex;"><span>	token <span style="color:#666">:=</span> mqttClient.<span style="color:#06287e">Publish</span>(topic, qos, retained, <span style="color:#007020">string</span>(<span style="color:#4070a0">&#34;on&#34;</span>))
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">if</span> token.<span style="color:#06287e">Wait</span>() <span style="color:#666">&amp;&amp;</span> token.<span style="color:#06287e">Error</span>() <span style="color:#666">!=</span> <span style="color:#007020;font-weight:bold">nil</span> {
</span></span><span style="display:flex;"><span>		<span style="color:#007020;font-weight:bold">return</span> fmt.<span style="color:#06287e">Errorf</span>(<span style="color:#4070a0">&#34;publishing to MQTT: %v&#34;</span>, token.<span style="color:#06287e">Error</span>())
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#007020;font-weight:bold">return</span> <span style="color:#007020;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><h2 id="conclusion">Conclusion</h2>
<p>I hope this small project write-up is useful to others in a similar situation!</p>
<p>If you need more features than that, check out the next step on the feature and
complexity ladder: <a href="https://pikvm.org/">PiKVM</a> or
<a href="https://tinypilotkvm.com/">TinyPilot</a>. See also <a href="https://www.jeffgeerling.com/blog/2021/raspberry-pi-kvms-compared-tinypilot-and-pi-kvm-v3">this comparison by Jeff
Geerling</a>.</p>


guid: https://michael.stapelberg.ch/posts/2023-01-15-gokrazy-instance-centric-config/
title: gokrazy: instance-centric configuration released 🎉
url: https://michael.stapelberg.ch/posts/2023-01-15-gokrazy-instance-centric-config/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p><a href="https://gokrazy.org/">gokrazy</a> is an appliance platform for Go programs: with
just a few commands, you can deploy your Go program(s) on a Raspberry Pi or a
(typically small) PC.</p>
<p>I’m excited to let you know that gokrazy now comes with a re-designed <code>gok</code>
command line tool and gokrazy instance configuration mechanism!</p>
<h2 id="context-gokrazy-in-a-few-words">Context: gokrazy in a few words</h2>
<img src="gokrazy-logo.png" align="right" width="200" style="margin-left: 1.5em" alt="gokrazy logo">
<p>The traditional way to run Go software on a Raspberry Pi would be to install
Raspbian or some other Linux distribution onto the SD card, copy over your
program(s) and then maintain that installation (do regular updates).</p>
<p>I thought it would be nicer to run my Raspberry Pis such that <strong>only Go
software</strong> is run by the Linux kernel on it, without any traditional Linux
distribution programs like package managers or even the usual <a href="https://en.wikipedia.org/wiki/GNU_Core_Utilities">GNU Core
Utilities</a>.</p>
<p>gokrazy builds Go programs into a read-only SquashFS root file system
image. When that image is started on a Raspberry Pi, a minimal init system
supervises the Go programs, and a DHCP and NTP client configure the IP address
and synchronize the time, respectively. After the first installation, all
subsequent updates can be done over the network, with an A/B partitioning
scheme.</p>
<p>I use gokrazy to, for example:</p>
<ul>
<li>
<p><a href="https://github.com/stapelberg/scan2drive">Scan incoming paper mail into PDF files on Google
Drive</a>.</p>
</li>
<li>
<p>Connect to the internet using <a href="https://router7.org/">router7, my small home internet
router</a> written in Go, running on a <a href="/posts/2021-07-10-linux-25gbit-internet-router-pc-build/">fast router PC
build</a> that handles a
<a href="/posts/2022-04-23-fiber7-25gbit-upgrade/">25 Gbit/s Fiber To The Home
connection</a>.</p>
</li>
<li>
<p><a href="/posts/2021-01-10-mqtt-introduction/">Automate the lights in my home</a>, and
<a href="https://github.com/stapelberg/hmgo">control and monitor the heating</a>.</p>
</li>
<li>
<p>Offer <a href="https://gokrazy.org/packages/tailscale/">Tailscale access</a> to a
Raspberry Pi Zero 2 W in my home network to then send Wake On Lan (WOL)
packets before SSH&rsquo;ing into my normally-suspended computers. See also my post
<a href="/posts/2022-08-27-out-of-band-remote-console/">DIY out-of-band management: remote console server
(2022)</a>.</p>
</li>
</ul>
<h2 id="before-and-after">Before and after</h2>
<p>Previously, the concept of gokrazy instance configuration was only a
convention. Each gokrazy build was created using the <code>gokr-packer</code> CLI tool, and
configured by the packer’s command-line flags, parameters, config files in
<code>~/.config</code> and per-package config files in the current directory
(e.g. <code>flags/github.com/gokrazy/breakglass/flags.txt</code>).</p>
<p>Now, all gokrazy commands and tools understand the <code>--instance</code> flag (or <code>-i</code>
for short), which determines the directory from which the <a href="https://gokrazy.org/userguide/instance-config/">Instance
Config</a> is read. For a gokrazy
instance named “hello”, the default directory is <code>~/gokrazy/hello</code>, which
contains the <code>config.json</code> file.</p>
<h2 id="example-creating-an-instance-for-a-go-working-copy">Example: creating an instance for a Go working copy</h2>
<p>Let’s say I have <a href="https://github.com/evcc-io/evcc">the evcc repository</a> cloned
as <code>~/src/evcc</code>. evcc is an electric vehicle charge controller with PV
integration, written in Go.</p>
<p>Now I want to run <code>evcc</code> on my Raspberry Pi using gokrazy. First, I create a new
instance:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>% gok -i evcc new
</span></span><span style="display:flex;"><span>gokrazy instance configuration created in /home/michael/gokrazy/evcc/config.json
</span></span><span style="display:flex;"><span><span style="color:#666">(</span>Use <span style="color:#4070a0">&#39;gok -i evcc edit&#39;</span> to edit the configuration interactively.<span style="color:#666">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Use <span style="color:#4070a0">&#39;gok -i evcc add&#39;</span> to add packages to this instance
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>To deploy this gokrazy instance, see <span style="color:#4070a0">&#39;gok help overwrite&#39;</span>
</span></span></code></pre></div><p>Now let’s add our working copy of <code>evcc</code> to the instance:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>% gok -i evcc add .
</span></span><span style="display:flex;"><span>2023/01/15 18:55:39 Adding the following package to gokrazy instance <span style="color:#4070a0">&#34;evcc&#34;</span>:
</span></span><span style="display:flex;"><span>  Go package  : github.com/evcc-io/evcc
</span></span><span style="display:flex;"><span>  in Go module: github.com/evcc-io/evcc
</span></span><span style="display:flex;"><span>  in <span style="color:#007020">local</span> dir: /tmp/evcc
</span></span><span style="display:flex;"><span>2023/01/15 18:55:39 Creating gokrazy builddir <span style="color:#007020;font-weight:bold">for</span> package github.com/evcc-io/evcc
</span></span><span style="display:flex;"><span>2023/01/15 18:55:39 Creating go.mod with replace directive
</span></span><span style="display:flex;"><span>go: creating new go.mod: module gokrazy/build/github.com/evcc-io/evcc
</span></span><span style="display:flex;"><span>2023/01/15 18:55:39 Adding package to gokrazy config
</span></span><span style="display:flex;"><span>2023/01/15 18:55:39 All <span style="color:#007020;font-weight:bold">done</span>! Next, use <span style="color:#4070a0">&#39;gok overwrite&#39;</span> <span style="color:#666">(</span>first deployment<span style="color:#666">)</span>, <span style="color:#4070a0">&#39;gok update&#39;</span> <span style="color:#666">(</span>following deployments<span style="color:#666">)</span> or <span style="color:#4070a0">&#39;gok run&#39;</span> <span style="color:#666">(</span>run on running instance temporarily<span style="color:#666">)</span>
</span></span></code></pre></div><p>We might want to monitor this Raspberry Pi’s stats later, so let’s add the
Prometheus node exporter to our gokrazy instance, too:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>% gok -i evcc add github.com/prometheus/node_exporter
</span></span><span style="display:flex;"><span>2023/01/15 19:04:05 Adding github.com/prometheus/node_exporter as a <span style="color:#666">(</span>non-local<span style="color:#666">)</span> package to gokrazy instance evcc
</span></span><span style="display:flex;"><span>2023/01/15 19:04:05 Creating gokrazy builddir <span style="color:#007020;font-weight:bold">for</span> package github.com/prometheus/node_exporter
</span></span><span style="display:flex;"><span>2023/01/15 19:04:05 Creating go.mod before calling go get
</span></span><span style="display:flex;"><span>go: creating new go.mod: module gokrazy/build/github.com/prometheus/node_exporter
</span></span><span style="display:flex;"><span>2023/01/15 19:04:05 running <span style="color:#666">[</span>go get github.com/prometheus/node_exporter@latest<span style="color:#666">]</span>
</span></span><span style="display:flex;"><span>go: downloading github.com/prometheus/node_exporter v1.5.0
</span></span><span style="display:flex;"><span><span style="color:#666">[</span>…<span style="color:#666">]</span>
</span></span><span style="display:flex;"><span>2023/01/15 19:04:07 Adding package to gokrazy config
</span></span></code></pre></div><p>It’s time to insert an SD card (<code>/dev/sdx</code>), which we will overwrite with a
gokrazy build:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>% gok -i evcc overwrite --full /dev/sdx
</span></span></code></pre></div><p>See <a href="https://gokrazy.org/quickstart/">gokrazy quickstart</a> for more detailed instructions.</p>
<h2 id="automation">Automation</h2>
<p>The new <code>gok</code> subcommands (<code>add</code>, <code>update</code>, etc.) are much easier to manage than
long <code>gokr-packer</code> command lines.</p>
<p>The new <a href="https://gokrazy.org/userguide/automation/">Automation page</a> shows how
to automate common tasks, be it daily updates via <code>cron</code>, or automated building
in Continuous Integration environments like GitHub Actions.</p>
<h2 id="migration">Migration</h2>
<p>Are you already a gokrazy user? If so, see <a href="https://gokrazy.org/userguide/migration-guide/">the Instance Config Migration
Guide</a> for how to switch from
the old <code>gokr-packer</code> tool to the new <code>gok</code> command.</p>
<h2 id="feedback--questions">Feedback / Questions?</h2>
<p>If you have any questions, please feel free to reach out at <a href="https://github.com/gokrazy/gokrazy/discussions">gokrazy GitHub
Discussions</a> 👋</p>


guid: https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/
title: Can Dell’s 6K monitor beat their 8K monitor?
url: https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>For the last 10 years, I have been interested in hi-DPI monitors, and recently I
read about an interesting new monitor: <a href="https://www.dell.com/de-ch/shop/dell-ultrasharp-32-6k-monitor-u3224kba/apd/210-bhnx/monitore-und-monitorzubeh%C3%B6r">Dell’s 32-inch 6K monitor
(U3224KBA)</a>,
a productivity monitor that offers plenty of modern connectivity options like
DisplayPort 2, HDMI 2 and Thunderbolt 4.</p>
<p>My current monitor is a <a href="/posts/2017-12-11-dell-up3218k/">Dell 32-inch 8K monitor
(UP3218K)</a>, which has a brilliant picture, but
a few annoying connectivity limitations and quirks — it needs two (!)
DisplayPort cables on a GPU with MST support, meaning that in practice, it only
works with nVidia graphics cards.</p>
<p>I was curious to try out the new 6K monitor to see if it would improve the
following points:</p>
<ul>
<li>Does the 6K monitor work well with most (all?) of my PCs and laptops?</li>
<li>Is 6K resolution enough, or would I miss the 8K resolution?</li>
<li>Is a matte screen the better option compared to the 8K monitor’s glossy finish?</li>
<li>Do the built-in peripherals work with Linux out of the box?</li>
</ul>
<p>I read <a href="https://www.heise.de/tests/32-Zoll-Display-mit-6K-und-USB-C-Dock-Dell-UltraSharp-U3224KBA-im-Test-9189751.html">a review on
heise+</a>
(also included in their c&rsquo;t magazine), but the review can’t answer these
subjective questions of mine.</p>
<p>So I ordered one and tried it out!</p>















<a href="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/IMG_2383_featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/IMG_2383_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_1638957_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/IMG_2383_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_1638957_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/IMG_2383_featured_hu855bbe0a9c47ac557e5b09db74fa3b06_1638957_600x0_resize_q75_box.jpg"
  
  width="600"
  height="450"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<h2 id="compatibility">Compatibility</h2>
<p>The native resolution of this monitor is 6144x3456 pixels.</p>
<p>To drive that resolution at 60 Hz, about 34 Gbps of data rate is needed.</p>
<p>DisplayPort 1.4a only offers a data rate of 25 Gbps, so your hardware and driver
need to support <a href="https://en.wikipedia.org/wiki/Display_Stream_Compression">Display Stream Compression
(DSC)</a> to reach the
full resolution at 60 Hz. I tried using DisplayPort 2.0, which supports 77 Gbps
of data rate, but the only GPU I have that supports DisplayPort 2 is the Intel
A380, which I could not get to work well with this monitor (see the next
section).</p>
<p>HDMI 2.1 offers 42 Gbps of data rate, but in my setup, the link would still
always use DSC.</p>
<p>Here are the combinations I have successfully tried:</p>
<table>
<thead>
<tr>
<th>Device</th>
<th>Cable</th>
<th>OS / Driver</th>
<th>Resolution</th>
</tr>
</thead>
<tbody>
<tr>
<td>MacBook Air M1</td>
<td>TB 3</td>
<td>macOS 13.4.1</td>
<td>native @ 60 Hz,<br> 8.1Gbps</td>
</tr>
<tr>
<td>GeForce RTX 4070<br>(DisplayPort 1.4a)</td>
<td>mDP-DP</td>
<td>Windows 11 21H2</td>
<td>native @ 60 Hz,<br> 12Gbps DSC</td>
</tr>
<tr>
<td>GeForce RTX 4070</td>
<td>mDP-DP</td>
<td>Linux 6.3<br>nVidia 535.54.03</td>
<td>native @ 60 Hz,<br> 8.1Gbps DSC</td>
</tr>
<tr>
<td>GeForce RTX 4070<br>(HDMI 2.1a)</td>
<td>HDMI</td>
<td>Windows 11 21H2</td>
<td>native @ 60 Hz,<br> 8.1Gbps DSC</td>
</tr>
<tr>
<td>GeForce RTX 4070</td>
<td>HDMI</td>
<td>Linux 6.3<br>nVidia 535.54.03</td>
<td>native @ 60 Hz,<br> 6Gbps 3CH DSC</td>
</tr>
<tr>
<td>GeForce RTX 3060</td>
<td>HDMI</td>
<td>Linux 6.3<br>nVidia 535.54.03</td>
<td>native @ 60 Hz,<br> 6Gbps 3CH DSC</td>
</tr>
<tr>
<td>ThinkPad X1 Extreme</td>
<td>TB 4</td>
<td>Linux 6.3<br>nVidia 535.54.03</td>
<td>native @ 60 Hz,<br> 8.1Gbps DSC</td>
</tr>
</tbody>
</table>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content"><strong>Note:</strong> on the ThinkPad X1 Extreme, I had to <a href="https://docs.fedoraproject.org/en-US/quick-docs/how-to-set-nvidia-as-primary-gpu-on-optimus-based-laptops/">set the nVidia GPU as primary
GPU</a>. When
the nVidia GPU is available, but routed through the Intel GPU, the native
resolution can be configured, but without hardware acceleration applications
like Chrome or Firefox are unusably slow.</div>
  </div>
</aside>

<p>The MacBook Air is the only device in my test that reaches full resolution
without using DSC.</p>
<h2 id="compatibility-issues">Compatibility issues</h2>
<p>Let’s talk about the combinations that did not work well.</p>
<h3 id="too-old-nvidia-driver--5355403-not-at-native-resolution">Too old nVidia driver (&lt; 535.54.03): not at native resolution</h3>
<p>You need a quite recent version of the nVidia driver, as they <strong>just recently</strong>
<a href="https://github.com/NVIDIA/open-gpu-kernel-modules/discussions/238">shipped support for
DSC</a> at high
resolutions. I successfully used DSC with 535.54.03.</p>
<p>With the “older” 530.41.03, I could only select 6016x3384 at 60 Hz, which is not
the native resolution of 6144x3456 at 60 Hz.</p>
<table>
<thead>
<tr>
<th>Device</th>
<th>Cable</th>
<th>OS / Driver</th>
<th>Resolution</th>
</tr>
</thead>
<tbody>
<tr>
<td>GeForce RTX 4070<br>(DisplayPort 1.4a)</td>
<td>mDP-DP</td>
<td>Linux 6.3<br>nVidia 530.41.03</td>
<td>native @ 30 Hz only,<br> 6016x3384@60</td>
</tr>
<tr>
<td>GeForce RTX 4070<br>(HDMI 2.1a)</td>
<td>HDMI</td>
<td>Linux 6.3<br>nVidia 530.41.03</td>
<td>native @ 30 Hz only,<br> 6016x3384@60</td>
</tr>
</tbody>
</table>
<h3 id="intel-gpu-no-picture-or-only-4k">Intel GPU: no picture or only 4K?!</h3>
<p>I was so excited when Intel announced that they are entering the graphics card
business. With all the experience and driver support for their integrated
graphics, I hoped for good Linux support.</p>
<p>Unfortunately, the Intel A380 I bought months ago continues to disappoint.</p>
<p>I could not get the 6K monitor to work at any resolution higher than 4K, not
even under Windows. Worse, when connecting the monitor using DisplayPort, I
wouldn’t get a picture at all (in Linux)!</p>
<table>
<thead>
<tr>
<th>Device</th>
<th>Cable</th>
<th>OS / Driver</th>
<th>Resolution</th>
</tr>
</thead>
<tbody>
<tr>
<td>ASRock Intel A380<br>(DisplayPort 2.0)</td>
<td>mDP-DP</td>
<td>Windows 11 21H2<br>Intel 31.0.101.4502</td>
<td>only 4K @ 60 Hz</td>
</tr>
<tr>
<td>ASRock Intel A380<br>(HDMI 2.0b)</td>
<td>HDMI</td>
<td>Windows 11 21H2<br>Intel 31.0.101.4502</td>
<td>only 4K @ 60 Hz</td>
</tr>
<tr>
<td>ASRock Intel A380<br>(DisplayPort 2.0)</td>
<td>mDP-DP</td>
<td>Linux 6.4</td>
<td>no picture in Xorg!</td>
</tr>
<tr>
<td>ASRock Intel A380<br>(HDMI 2.0b)</td>
<td>HDMI</td>
<td>Linux 6.4</td>
<td>only 4K @ 60 Hz</td>
</tr>
</tbody>
</table>
<h3 id="resume">No picture after resume from suspend-to-RAM</h3>
<p>I suspend my PC to RAM at least once per day, sometimes even more often.</p>
<p>With my current 8K monitor, I have nailed the suspend/wakeup procedure. With the
help of a smart plug, I’m automatically turning the monitor off (on suspend) and
on (on wakeup). After a couple of seconds of delay, I configure the correct
resolution using <code>xrandr</code>.</p>
<p>I had hoped that the 6K monitor would make any sort of intricate automation
superfluous.</p>
<p>Unfortunately, when I resumed my PC, I noticed that the monitor would not show a
picture at all! I had to log in from my laptop via SSH to change the resolution
with <code>xrandr</code> to 4K, then power the monitor off and on again, then change
resolution back to the native 6K.</p>
<h2 id="scaling">Scaling</h2>
<p>Once you have a physical connection established, how do you configure your
computer? With 6K at 32 inches, you’ll need to enable some kind of scaling in
order to comfortably read text.</p>
<p>This section shows what options Linux and macOS offer.</p>
<h3 id="i3-x11">i3 (X11)</h3>
<p>Just like many other programs on Linux, you configure i3’s scaling by <a href="https://wiki.archlinux.org/title/HiDPI#X_Resources">setting
the <code>Xft.dpi</code> X
resource</a>. The default is 96
dpi, so to get 200% scaling, set <code>Xft.dpi: 192</code>.</p>
<p>Personally, I found 240% scaling more comfortable, i.e. <code>Xft.dpi: 230</code>.</p>
<p>This corresponds to a logical resolution of 2560x1440 pixels.</p>
<h3 id="gnome-wayland">GNOME (Wayland)</h3>
<p>I figured I’d also give Wayland a shot, so I ran GNOME in Fedora 38 on my
ThinkPad X1 Extreme.</p>
<p>Here’s what the settings app shows in its “Displays” tab:</p>















<a href="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/Screenshot%20from%202023-07-02%2012-25-57.png"><img
  srcset="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/Screenshot%20from%202023-07-02%2012-25-57_hu2cabb68d8c47fd27ed6a38a3c2c68dd8_125601_1200x0_resize_box_3.png 2x,https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/Screenshot%20from%202023-07-02%2012-25-57_hu2cabb68d8c47fd27ed6a38a3c2c68dd8_125601_1800x0_resize_box_3.png 3x"
  src="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/Screenshot%20from%202023-07-02%2012-25-57_hu2cabb68d8c47fd27ed6a38a3c2c68dd8_125601_600x0_resize_box_3.png"
  
  width="600"
  height="415"
  style="

border: 0;

"
  
  loading="lazy"></a>



<p>I tried <a href="https://www.omglinux.com/how-to-enable-fractional-scaling-fedora/">enabling fractional
scaling</a>, but
then GNOME froze until I disconnected the Dell monitor.</p>
<h3 id="macos">macOS</h3>
<p>When connecting the monitor to my MacBook Air M1 (2020), it defaults to a
logical resolution of 3072x1728, i.e. 200% scaling.</p>















<a href="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/2023-07-02-macOS-displays.png"><img
  srcset="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/2023-07-02-macOS-displays_hubf6579ec96f99144ce4f6a52859f99df_573205_1200x0_resize_box_3.png 2x,https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/2023-07-02-macOS-displays_hubf6579ec96f99144ce4f6a52859f99df_573205_1800x0_resize_box_3.png 3x"
  src="https://michael.stapelberg.ch/posts/2023-07-03-dell-u3224kba-32-inch-6k-monitor/2023-07-02-macOS-displays_hubf6579ec96f99144ce4f6a52859f99df_573205_600x0_resize_box_3.png"
  
  width="600"
  height="617"
  style="

border: 0;

"
  
  loading="lazy"></a>



<p>For comparison, with <a href="https://www.apple.com/studio-display/">Apple’s (5K) Studio
Display</a>, the default setting is
2560x1440 (200% scaling), or 2880x1620 (“More Space”, 177% scaling).</p>
<h2 id="observations">Observations</h2>
<h3 id="matte-screen">Matte screen</h3>
<p>I remember the uproar when Lenovo introduced ThinkPads with glossy screens. At
the time, I thought I prefer matte screens, but over the years, I heard that
glossy screens are getting better and better, and consumers typically prefer
them for their better picture quality.</p>
<p>The 8K monitor I’m using has a glossy screen on which reflections are quite
visible. The MacBook Air’s screen shows fewer reflections in comparison.</p>
<p>Dell’s 6K monitor offers me a nice opportunity to see which option I prefer.</p>
<p>Surprisingly, I found that I don’t like the matte screen better!</p>
<p>It’s hard to describe, but somehow the picture seems more “dull”, or less bright
(independent of the actual brightness of the monitor), or more toned down. The
colors don’t pop as much.</p>
<h3 id="philosophical-question-peripherals-powered-on-by-default">Philosophical question: peripherals powered on by default?</h3>
<p>One thing that I did not anticipate beforehand is the difference in how
peripherals are treated when they are built into the monitor vs. when they are
plugged into a USB hub.</p>
<p>I like to have my peripherals off-by-default, with “on” being the exceptional
state. In fact, I leave my microphone disconnected and only plug its USB cable
in when I need it. I also recently realized that I want sound to only be played
on headphones, so I disconnected my normal speakers in favor of my Bluetooth
dongle.</p>
<p>The 6K monitor, on the other hand, has all of its peripherals on-by-default, and
bright red LEDs light up when the speaker or microphone is muted.</p>
<p>This is the opposite of how I want my peripherals to behave, but of course I
understand why Dell developed the monitor with on-by-default peripherals.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Let’s go back to the questions I started the article with and answer them one by one:</p>
<ol>
<li>
<p>Does the 6K monitor work well with most (all?) of my PCs and laptops?</p>
<p>→ <strong>Answer:</strong> The 6K monitor works a lot better than the 8K monitor, but that’s a
low bar to clear. I would still call the 6K monitor finicky. Even when you
run a latest-gen GPU with latest drivers, the monitor does not reliably show
a picture after a suspend/resume cycle.</p>
</li>
<li>
<p>Is 6K resolution enough, or would I miss the 8K resolution?</p>
<p>→ <strong>Answer:</strong> I had really hoped that 6K would turn out to be enough, but the
difference to 8K is visible with the bare eye. Just like 200% scaling is a
nice step up from working at 96 dpi, 300% scaling (what I use on 8K) is
another noticeable step up.</p>
</li>
<li>
<p>Is a matte screen the better option compared to the 8K monitor’s glossy finish?</p>
<p>→ <strong>Answer:</strong> While I don’t like the reflections in Dell’s 8K monitor, the
picture quality is undeniably better compared to a matte screen. The 6K
monitor just doesn’t look as good, and it’s not just about the difference in
text sharpness.</p>
</li>
<li>
<p>Do the built-in peripherals work with Linux out of the box?</p>
<p>→ <strong>Answer:</strong> Yes, as far as I can tell. The webcam works fine with the
generic <code>uvcvideo</code> USB webcam driver, the microphone and speakers work out of
the box. I have not tested the presence sensor.</p>
</li>
</ol>
<p>So, would I recommend the monitor? Depends on what you’re using as your current
monitor and as the device you want to connect!</p>
<p>If you’re coming from a 4K display, the 6K resolution will be a nice step
up. Connecting a MacBook Air M1 or newer is a great experience. If you want to
connect PCs, be sure to use a new-enough nVidia GPU with latest drivers. Even
under these ideal conditions, you might run into quirks like the <a href="#resume">no picture
after resume</a> problem. If you don’t mind early adopter pains like that,
and are looking for a monitor that includes peripherals, go for it!</p>
<p>For me, switching from my 8K monitor would be a downgrade without enough
benefits.</p>
<p>The ideal monitor for me would be a mixture between Dell’s 8K and 6K models:</p>
<ul>
<li>8K resolution
<ul>
<li>…but with more modern connectivity options (one cable! works out of the box!).</li>
</ul>
</li>
<li>without built-in peripherals like webcam, microphone and speaker
<ul>
<li>…but with the USB KVM switch concept (monitor input coupled to USB upstream).</li>
</ul>
</li>
<li>glossy finish for best picture quality
<ul>
<li>…but with fewer reflections.</li>
</ul>
</li>
</ul>
<p>Maybe they’ll develop an updated version of the 8K monitor at some point?</p>


guid: https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/
title: My 2023 all-flash ZFS NAS (Network Storage) build
url: https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>For over 10 years now, I run two self-built NAS (Network Storage) devices which serve media (currently via Jellyfin) and run daily backups of all my PCs and servers.</p>
<p>In this article, I describe my goals, which hardware I picked for my new build (and why) and how I set it up.</p>
<h2 id="design-goals">Design Goals</h2>
<p>I use my network storage devices primarily for archival (daily backups), and secondarily as a media server.</p>
<p>There are days when I don’t consume any media (TV series and movies) from my NAS, because I have my music collection mirrored to another server that’s running 24/7 anyway. In total, my NAS runs for a few hours in some evenings, and for about an hour (daily backups) in the mornings.</p>
<p>This usage pattern is distinctly different than, for example, running a NAS as a file server for collaborative video editing that needs to be available 24/7.</p>
<p>The goals of my NAS setup are:</p>
<ol>
<li>Save power: each NAS build only runs when needed.
<ul>
<li>They must support Wake-on-LAN or <a href="/posts/2022-10-09-remote-power-button/">similar (ESP32 remote power button)</a>.</li>
<li>Scheduling of backups is done separately, on a Raspberry Pi with <a href="https://gokrazy.org/">gokrazy</a>.</li>
<li>Convenient <a href="https://github.com/stapelberg/regelwerk/commit/8b81d7a808b1d76a0e96bdb4ab43964623d133c4">power off (tied to our all-lights-out button)</a> and power on (with <a href="https://github.com/stapelberg/zkj-nas-tools/blob/master/webwake/webwake.go">webwake</a>).</li>
</ul>
</li>
<li>Use Off-the-shelf hardware and software.
<ul>
<li>When hardware breaks, I can get replacements from the local PC store the same day.</li>
<li>Even when only the data disk(s) survive, I should be able to access my data when booting a standard live Linux system.</li>
<li>Minimal application software risk: I want to minimize risk for manual screw-ups or software bugs, meaning I use the venerable rsync for my backup needs (not Borg, restic, or similar).</li>
<li>Minimal system software risk: I use reliable file systems with the minimal feature set — no LVM or btrfs snapshots, no ZFS replication, etc. To achieve redundancy, I don’t use a cluster file system with replication, instead I synchronize my two NAS builds using rsync, without the <code>--delete</code> flag.</li>
</ul>
</li>
<li>Minimal failure domains: when one NAS fails, the other one keeps working.
<ul>
<li>Having N+1 redundancy here takes the stress out of repairing your NAS.</li>
<li>I run each NAS in a separate room, so that accidents like fires or spilled drinks only affect one machine.</li>
</ul>
</li>
</ol>
<h4 id="file-system-zfs">File System: ZFS</h4>
<p>In this specific build, I am trying out <a href="https://en.wikipedia.org/wiki/ZFS">ZFS</a>. Because I have two NAS builds
running, it is easy to change one variable of the system (which file system to
use) in one build, without affecting the other build.</p>
<p>My main motivation for using ZFS instead of <a href="https://en.wikipedia.org/wiki/Ext4"><code>ext4</code></a> is that ZFS does data checksumming, whereas ext4 only checksums metadata and the journal, but not data at rest. With large enough datasets, the chance of bit flips increases significantly, and I would prefer to know about them so that I can restore the affected files from another copy.</p>
<h2 id="hardware">Hardware</h2>
<p>Each of the two storage builds has (almost) the same components. This makes it easy to diagnose one with the help of the other. When needed, I can swap out components of the second build to temporarily repair the first one, or vice versa.</p>















<a href="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/IMG_1974.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/IMG_1974_hubee9c905cc3b7237e7c96518cbb38b46_1156723_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/IMG_1974_hubee9c905cc3b7237e7c96518cbb38b46_1156723_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/IMG_1974_hubee9c905cc3b7237e7c96518cbb38b46_1156723_600x0_resize_q75_box.jpg"
  alt="photo of the Network Storage PC from the side, showing the Noctua case fan and CPU cooler, data disks, PSU and cables" title="photo of the Network Storage PC from the side, showing the Noctua case fan and CPU cooler, data disks, PSU and cables"
  width="600"
  height="450"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<h3 id="base-components">Base Components</h3>
<table>
<thead>
<tr>
<th>Price</th>
<th>Type</th>
<th>Article</th>
<th>Remark</th>
</tr>
</thead>
<tbody>
<tr>
<td>114 CHF</td>
<td>mainboard</td>
<td><a href="https://www.digitec.ch/en/s1/product/asrock-b450-gaming-itxac-am4-amd-b450-mini-itx-motherboards-9385702">AsRock B450 Gaming ITX/ac</a></td>
<td>Mini ITX</td>
</tr>
<tr>
<td>80 CHF</td>
<td>cpu</td>
<td><a href="https://www.heise.de/preisvergleich/amd-athlon-3000g-yd3000c6m2ofh-yd3000c6fhmpk-a2174924.html?hloc=at&amp;hloc=de">AMD Athlon 3000G</a></td>
<td>35W TDP, GPU</td>
</tr>
<tr>
<td>65 CHF</td>
<td>cpu cooler</td>
<td><a href="https://www.digitec.ch/de/s1/product/noctua-nh-l12s-70-mm-cpu-kuehler-6817433">Noctua NH-L12S</a></td>
<td>silent!</td>
</tr>
<tr>
<td>58 CHF</td>
<td>power supply</td>
<td><a href="https://www.digitec.ch/en/s1/product/silverstone-power-supply-st30sf-300w-sfx-300-w-power-supply-pc-5988297">Silverstone ST30SF 300W SFX</a></td>
<td>SFX form factor</td>
</tr>
<tr>
<td>51 CHF</td>
<td>case</td>
<td><a href="https://www.digitec.ch/en/s1/product/silverstone-sst-sg05bb-lite-mini-itx-mini-dtx-pc-case-3525365">Silverstone SST-SG05BB-Lite</a></td>
<td>Mini ITX</td>
</tr>
<tr>
<td>48 CHF</td>
<td>system disk</td>
<td><a href="https://www.digitec.ch/en/s1/product/wd-red-sn700-250-gb-m2-2280-ssd-17688689">WD Red SN700 250GB</a></td>
<td>M.2 NVMe</td>
</tr>
<tr>
<td>32 CHF</td>
<td>case fan</td>
<td><a href="https://www.digitec.ch/en/s1/product/noctua-nf-s12a-uln-120mm-1x-pc-fans-2451401">Noctua NF-S12A ULN</a></td>
<td>silent 120mm</td>
</tr>
<tr>
<td>28 CHF</td>
<td>ram</td>
<td><a href="https://www.digitec.ch/en/s1/product/gskill-value-1-x-8gb-2400-mhz-ddr4-ram-dimm-ram-11056524">8 GB DDR4 Value RAM (F4-2400C15-8GNT)</a></td>
<td></td>
</tr>
</tbody>
</table>
<p>The total price of 476 CHF makes this not a cheap build.</p>
<p>But, I think each component is well worth its price. Here’s my thinking regarding the components:</p>
<ul>
<li>Why not a cheaper <strong>system disk</strong>? I wanted to use an M.2 NVMe disk so that I could mount it on the bottom of the mainboard instead of having to mount another SATA disk in the already-crowded case. Instead of chosing the cheapest M.2 disk I could find, I went with WD Red as a brand I recognize. While it’s not a lot of effort to re-install the system disk, it’s still annoying and something I want to avoid if possible. If spending 20 bucks saves me one disk swap + re-install, that’s well worth it for me!</li>
<li>Why not skip the <strong>system disk</strong> entirely and install on the data disks? That makes the system harder to (re-)install, and easier to make manual errors when recovering the system. I like to physically disconnect the data disks while re-installing a NAS, for example. (I’m a fan of simple precautions that prevent drastic mistakes!)</li>
<li>Why not a cheaper <strong>CPU cooler</strong>? In <a href="/posts/2019-10-23-nas/">one of my earlier NAS builds</a>, I used a (cheaper) passive CPU fan, which was directly in the air stream of the Noctua 120mm case fan. This setup was spec&rsquo;ed for the CPU I used, and yet said CPU died as the only CPU to die on me in many many years. I want a reliable CPU fan, but also an absolutely silent build, so I went with the Noctua CPU cooler.</li>
<li>Why not skip the <strong>case fan</strong>, or go with the Silverstone-supplied one? You might argue that the airflow of the CPU cooler is sufficient for this entire build. Maybe that’s true, but I don’t want to risk it. Also, there are 3 disks (two data disks and one system disk) that can benefit from additional airflow.</li>
<li>Regarding the <strong>CPU</strong>, I chose the cheapest AMD CPU for Socket AM4, with a 35W TDP and built-in graphics. The built-in graphics means I can connect an HDMI monitor for setup and troubleshooting, without having to use the mainboard’s valuable one and only PCIe slot.
<br>
<br>
Unfortunately, AMD CPUs with 35W TDP are not readily available right now. My tip is to look around for a bit, and maybe buy a used one. Chose either the predecessor Athlon 200GE, or the newer generation Ryzen APU series, whichever you can get your hands on.</li>
<li>Regarding the <strong>mainboard</strong>, I went with the AsRock Mini ITX series, which have served me well over the years. I started with an <a href="https://www.asrock.com/mb/AMD/AM1H-itx/">AsRock AM1H-ITX</a> in 2016, then bought two <a href="https://www.digitec.ch/en/s1/product/asrock-ab350-gaming-itxac-am4-amd-b350-mini-itx-motherboards-7022839">AsRock AB350 Gaming ITX/ac</a> in 2019, and recently an <a href="https://www.digitec.ch/en/s1/product/asrock-b450-gaming-itxac-am4-amd-b450-mini-itx-motherboards-9385702">AsRock B450 Gaming ITX/ac</a>.</li>
</ul>
<p>As a disclaimer: the two builds I use are <em>very similar</em> to the component list above, with the following differences:</p>
<ol>
<li>On storage2, I use an old AMD Ryzen 5 5600X CPU instead of the listed Athlon 3000G. The extra performance isn’t needed, and the lack of integrated graphics is annoying. But, I had the CPU lying around and didn’t want it to go to waste.</li>
<li>On storage3, I use an old AMD Athlon 200GE CPU on an <a href="https://www.digitec.ch/en/s1/product/asrock-ab350-gaming-itxac-am4-amd-b350-mini-itx-motherboards-7022839">AsRock AB350</a> mainboard.</li>
</ol>
<p>I didn’t describe the <em>exact</em> builds I use because a component list is more useful if the components on it are actually available :-).</p>
<h3 id="16-tb-ssd-data-disks">16 TB SSD Data Disks</h3>
<p>It used to be that Solid State Drives (SSDs) were just way too expensive compared to spinning hard disks when talking about terabyte sizes, so I used to put the largest single disk drive I could find into each NAS build: I started with 8 TB disks, then upgraded to 16 TB disks later.</p>
<p>Luckily, the price of flash storage has come down quite a bit: the <a href="https://www.digitec.ch/en/s1/product/samsung-870-qvo-8000-gb-25-ssd-13388185">Samsung SSD 870 QVO (8 TB)</a> costs “only” 42 CHF per TB. For a total of 658 CHF, I can get 16 TB of flash storage in 2 drives:</p>















<a href="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-22-samsung-870qvo-featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-22-samsung-870qvo-featured_hu799c805b17c89c62874113693200acea_520811_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-22-samsung-870qvo-featured_hu799c805b17c89c62874113693200acea_520811_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-22-samsung-870qvo-featured_hu799c805b17c89c62874113693200acea_520811_600x0_resize_q75_box.jpg"
  alt="two samsung 870 QVO disks" title="two samsung 870 QVO disks"
  width="600"
  height="338"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<p>Of course, spinning hard disks are at 16 CHF per TB, so going all-flash is over 3x as expensive.</p>
<p>I decided to pay the premium to get a number of benefits:</p>
<ul>
<li>My NAS devices are quieter because there are no more spinning disks in them. This gives me more flexibility in where to physically locate each storage machine.</li>
<li>My daily backups run quicker, meaning each NAS needs to be powered on for less time. The effect was actually quite pronounced, because figuring out which files need backing up requires a lot of random disk access. My backups used to take about 1 hour, and now finish in less than 20 minutes.</li>
<li>The quick access times of SSDs solve the last remaining wrinkle in my backup scheme: deleting backups and measuring used disk space is finally fast!</li>
</ul>
<h3 id="power-usage">Power Usage</h3>
<p>The choice of CPU, Mainboard and Network Card all influence the total power usage of the system. Here are a couple of measurements to give you a rough idea of the power usage:</p>
<table>
<thead>
<tr>
<th>build</th>
<th>CPU</th>
<th>main board</th>
<th>network card</th>
<th>idle</th>
<th>load</th>
</tr>
</thead>
<tbody>
<tr>
<td>s2</td>
<td>5600X</td>
<td><a href="https://www.digitec.ch/en/s1/product/asrock-b450-gaming-itxac-am4-amd-b450-mini-itx-motherboards-9385702">B450</a></td>
<td>10G: Mellanox ConnectX-3</td>
<td>26W</td>
<td>60W</td>
</tr>
<tr>
<td>s3</td>
<td>200GE</td>
<td><a href="https://www.digitec.ch/en/s1/product/asrock-ab350-gaming-itxac-am4-amd-b350-mini-itx-motherboards-7022839">AB350</a></td>
<td>10G: <a href="https://www.fs.com/products/135978.html">FS Intel 82599</a></td>
<td>28W</td>
<td>50W</td>
</tr>
<tr>
<td>s3</td>
<td>200GE</td>
<td><a href="https://www.digitec.ch/en/s1/product/asrock-ab350-gaming-itxac-am4-amd-b350-mini-itx-motherboards-7022839">AB350</a></td>
<td>1G onboard</td>
<td>23W</td>
<td>40W</td>
</tr>
</tbody>
</table>
<p>These values were measured using a <a href="https://mystrom.ch/de/wifi-switch/">myStrom WiFi Switch</a>.</p>
<h2 id="operating-system">Operating System</h2>
<h3 id="previously-coreos">Previously: CoreOS</h3>
<p>Before this build, I ran my NAS using Docker containers on <a href="https://en.wikipedia.org/wiki/Container_Linux">CoreOS (later renamed to Container Linux)</a>, which was a light-weight Linux distribution focused on containers. There are two parts about CoreOS that I liked most.</p>
<p>The most important part was that CoreOS updated automatically, using an A/B updating scheme, just like I do in <a href="https://gokrazy.org/">gokrazy</a>. I want to run as many of my devices as possible with A/B updates.</p>
<p>The other bit I like is that the configuration is very clearly separated from the OS. I managed the configuration (a <a href="https://cloud-init.io/">cloud-init YAML file</a>) on my main PC, so when swapping out the NAS system disk with a blank disk, I could just plug my config file into the CoreOS installer, and be done.</p>
<p>When CoreOS was bought by Red Hat and merged into Project Atomic, there wasn’t a good migration path and cloud-init wasn’t supported anymore. As a short-term solution, I switched from CoreOS to Flatcar Linux, a spiritual successor.</p>
<h3 id="now-ubuntu-server">Now: Ubuntu Server</h3>
<p>For this build, I wanted to try out ZFS. I always got the impression that ZFS was a pain to run because its kernel modules are not included in the upstream Linux kernel source.</p>
<p>Then, in 2016, Ubuntu decided to include ZFS by default. There are a couple of other Linux distributions on which ZFS seems easy enough to run, like Gentoo, Arch Linux or NixOS.</p>
<p>I wanted to spend my “innovation tokens” on ZFS, and keep the rest boring and similar to what I already know and work with, so I chose Ubuntu Server over NixOS. It’s similar enough to Debian that I don’t need to re-learn.</p>
<p>Luckily, the migration path from Flatcar’s cloud-init config to Ubuntu Server is really easy: just copy over parts of the cloud-config until you’re through the entire thing. It’s like a checklist!</p>
<h3 id="maybe-later-gokrazy">Maybe later? gokrazy</h3>
<p>In the future, it might be interesting to build a NAS setup using <a href="https://gokrazy.org">gokrazy</a>. In particular since we now can <a href="https://gokrazy.org/packages/docker-containers/">run Docker containers on gokrazy</a>, which makes running Samba or Jellyfin quite easy!</p>
<p>Using gokrazy instead of Ubuntu Server would get rid of a lot of moving parts. The current blocker is that ZFS is not available on gokrazy. Unfortunately that’s not easy to change, in particular also from a licensing perspective.</p>
<h2 id="setup">Setup</h2>
<h3 id="uefi">UEFI</h3>
<p>I changed the following UEFI settings:</p>
<ul>
<li>
<p>Advanced → ACPI Configuration → PCIE Devices Power On: Enabled</p>
<ul>
<li>This setting is needed (but not sufficient) for Wake On LAN (WOL). You also need to enable WOL in your operating system.</li>
</ul>
</li>
<li>
<p>Advanced → Onboard Devices Configuration → Restore on AC/Power Loss: Power On</p>
<ul>
<li>This setting ensures the machine turns back on after a power loss. Without it, WOL might not work after a power loss.</li>
</ul>
</li>
</ul>
<h3 id="operating-system-1">Operating System</h3>
<h4 id="network-preparation">Network preparation</h4>
<p>I like to configure static IP addresses for devices that are a permanent part of my network.</p>
<p>I have come to prefer configuring static addresses as static DHCP leases in my router, because then the address remains the same no matter which operating system I boot — whether it’s the installed one, or a live USB stick for debugging.</p>
<h4 id="ubuntu-server">Ubuntu Server</h4>
<ol>
<li>
<p>Download Ubuntu Server from <a href="https://ubuntu.com/download/server">https://ubuntu.com/download/server</a></p>
<ul>
<li>I initially let the setup program install Docker, but that’s a mistake. The setup program will get you Docker from snap (not apt), which <a href="https://stackoverflow.com/questions/52526219/docker-mkdir-read-only-file-system">can’t work with the whole file system</a>.</li>
</ul>
</li>
<li>
<p>Disable swap:</p>
<ul>
<li><code>swapoff -a</code></li>
<li><code>$EDITOR /etc/fstab</code> # delete the swap line</li>
</ul>
</li>
<li>
<p>Automatically load the corresponding sensors kernel module for the mainboard so that the Prometheus node exporter picks up temperature values and fan speed values:</p>
<ul>
<li><code>echo nct6775 | sudo tee /etc/modules</code></li>
</ul>
</li>
<li>
<p>Enable <a href="https://help.ubuntu.com/community/AutomaticSecurityUpdates">unattended upgrades</a>:</p>
<ul>
<li>
<p><code>dpkg-reconfigure -plow unattended-upgrades</code></p>
</li>
<li>
<p>Edit <code>/etc/apt/apt.conf.d/50unattended-upgrades</code> — I like to make the following changes:</p>
<pre tabindex="0"><code>Unattended-Upgrade::MinimalSteps &#34;true&#34;;
Unattended-Upgrade::Mail &#34;michael@example.net&#34;;
Unattended-Upgrade::MailReport &#34;only-on-error&#34;;
Unattended-Upgrade::Automatic-Reboot &#34;true&#34;;
Unattended-Upgrade::Automatic-Reboot-Time &#34;08:00&#34;;
Unattended-Upgrade::SyslogEnable &#34;true&#34;;
</code></pre></li>
</ul>
</li>
</ol>
<h3 id="network">Network</h3>
<h4 id="tailscale-mesh-vpn">Tailscale Mesh VPN</h4>
<p>I have come to like Tailscale. It’s a mesh VPN (data flows directly between the machines) that allows me access to and from my PCs, servers and storage machines from anywhere.</p>
<p>Specifically, I followed the <a href="https://tailscale.com/download/linux/ubuntu-2204">install Tailscale on Ubuntu 22.04 guide</a>.</p>
<h4 id="prometheus-node-exporter">Prometheus Node Exporter</h4>
<p>For monitoring, I have an existing Prometheus setup. To add a new machine to my setup, I need to configure it as a new target on my Prometheus server. In addition, I need to set up Prometheus on the new machine.</p>
<p>First, I installed the Prometheus node exporter using <code>apt install prometheus-node-exporter</code>.</p>
<p>Then, I modified <code>/etc/default/prometheus-node-exporter</code> to only listen on the Tailscale IP address:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#bb60d5">ARGS</span><span style="color:#666">=</span><span style="color:#4070a0">&#34;--web.listen-address=100.85.3.16:9100&#34;</span>
</span></span></code></pre></div><p>Lastly, I added a systemd override to ensure the node exporter keeps trying to start until tailscale is up: the command <code>systemctl edit prometheus-node-exporter</code> opens an editor, and I configured the override like so:</p>
<pre tabindex="0"><code># /etc/systemd/system/prometheus-node-exporter.service.d/override.conf
[Unit]
# Allow infinite restarts, even within a short time.
StartLimitIntervalSec=0

[Service]
RestartSec=1
</code></pre><h4 id="static-ipv6-address">Static IPv6 address</h4>
<p>Similar to the static IPv4 address, I like to give my NAS a static IPv6 address as well. This way, I don’t need to reconfigure remote systems when I (sometimes temporarily) switch my NAS to a different network card with a different MAC address. Of course, this point becomes moot if I ever switch all my backups to Tailscale.</p>
<p>Ubuntu Server comes with Netplan by default, but I don’t know Netplan and don’t want to use it.</p>
<p>To switch to <code>systemd-networkd</code>, I ran:</p>
<pre tabindex="0"><code>apt remove --purge netplan.io
systemctl enable --now systemd-networkd
</code></pre><p>Then, I created a <code>systemd-networkd</code> config file with a static IPv6 token, resulting in a predictable IPv6 address:</p>
<pre tabindex="0"><code>$EDITOR /etc/systemd/network/enp.network
</code></pre><p>My config file looks like this:</p>
<pre tabindex="0"><code>[Match]
Name=enp*

[Network]
DHCP=yes
IPv6Token=0:0:0:0:10::253
IPv6AcceptRouterAdvertisements=yes
</code></pre><h4 id="ipv6-firewall-setup">IPv6 firewall setup</h4>
<p>An easy way to configure Linux’s <code>netfilter</code> firewall is to <code>apt install iptables-persistent</code>. That package takes care of saving firewall rules on shutdown and restoring them on the next system boot.</p>
<p>My rule setup is very simple: allow ICMP (IPv6 needs it), then set up <code>ACCEPT</code> rules for the traffic I expect, and <code>DROP</code> the rest.</p>
<p>Here’s my resulting <code>/etc/iptables/rules.v6</code> from such a setup:</p>
<details>
<summary>
<code>/etc/iptables/rules.v6</code>
</summary>
<pre tabindex="0"><code># Generated by ip6tables-save v1.4.14 on Fri Aug 26 19:57:51 2016
*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p ipv6-icmp -m comment --comment &#34;IPv6 needs ICMPv6 to work&#34; -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -m comment --comment &#34;Allow packets for outgoing connections&#34; -j ACCEPT
-A INPUT -s fe80::/10 -d fe80::/10 -m comment --comment &#34;Allow link-local traffic&#34; -j ACCEPT
-A INPUT -s 2001:db8::/64 -m comment --comment &#34;local traffic&#34; -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m comment --comment &#34;SSH&#34; -j ACCEPT
COMMIT
# Completed on Fri Aug 26 19:57:51 2016
</code></pre></details>
<h3 id="encrypted-zfs">Encrypted ZFS</h3>
<p>Before you can use ZFS, you need to install the ZFS tools using <code>apt install zfsutils-linux</code>.</p>
<p>Then, we create a zpool that spans both SSDs:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>zpool create <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">ashift</span><span style="color:#666">=</span><span style="color:#40a070">12</span> <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  srv <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  /dev/disk/by-id/ata-Samsung_SSD_870_QVO_8TB_S5SSNF0TC06121Z <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  /dev/disk/by-id/ata-Samsung_SSD_870_QVO_8TB_S5SSNF0TC06787P
</span></span></code></pre></div><p>The <code>-o ashift=12</code> ensures <a href="https://wiki.archlinux.org/title/ZFS#Advanced_Format_disks">proper alignment</a> on disks with a sector size of either 512B or 4KB.</p>
<p>On that zpool, we now create our datasets:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#666">(</span><span style="color:#007020">echo</span> -n on-device-secret <span style="color:#666">&amp;&amp;</span> <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span> wget -qO - https://autounlock.zekjur.net:8443/nascrypto<span style="color:#666">)</span> | zfs create <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">encryption</span><span style="color:#666">=</span>on <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">compression</span><span style="color:#666">=</span>off <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">atime</span><span style="color:#666">=</span>off <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">keyformat</span><span style="color:#666">=</span>passphrase <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  -o <span style="color:#bb60d5">keylocation</span><span style="color:#666">=</span>file:///dev/stdin <span style="color:#4070a0;font-weight:bold">\
</span></span></span><span style="display:flex;"><span><span style="color:#4070a0;font-weight:bold"></span>  srv/data
</span></span></code></pre></div><p>The key I’m piping into <code>zfs create</code> is constructed from two halves: the on-device secret and the remote secret, which is a setup I’m using to implement an automated crypto unlock that is remotely revokable. See the next section for the corresponding <code>unlock.service</code>.</p>
<p>I repeated this same command (adjusting the dataset name) for each dataset: I currently have one for <code>data</code> and one for <code>backup</code>, just so that the used disk space of each major use case is separately visible:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>df -h /srv /srv/backup /srv/data   
</span></span><span style="display:flex;"><span>Filesystem      Size  Used Avail Use% Mounted on
</span></span><span style="display:flex;"><span>srv             4,2T  128K  4,2T   1% /srv
</span></span><span style="display:flex;"><span>srv/backup      8,1T  3,9T  4,2T  49% /srv/backup
</span></span><span style="display:flex;"><span>srv/data         11T  6,4T  4,2T  61% /srv/data
</span></span></code></pre></div><h4 id="zfs-maintenance">ZFS maintenance</h4>
<p>To detect errors on your disks, ZFS has a feature called “scrubbing”. I don’t think I need to scrub more often than monthly, but <a href="https://wiki.archlinux.org/title/ZFS#Scrubbing">maybe your scrubbing requirements are different</a>.</p>
<p>I enabled monthly scrubbing on my zpool <code>srv</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>systemctl <span style="color:#007020">enable</span> --now zfs-scrub-monthly@srv.timer
</span></span></code></pre></div><p>On this machine, a scrub takes a little over 4 hours and keeps the disks busy:</p>
<pre tabindex="0"><code>  scan: scrub in progress since Wed Oct 11 16:32:05 2023
	808G scanned at 909M/s, 735G issued at 827M/s, 10.2T total
	0B repaired, 7.01% done, 03:21:02 to go
</code></pre><p>We can confirm by looking at the Prometheus Node Exporter metrics:</p>















<a href="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-11-grafana-scrub.png"><img
  srcset="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-11-grafana-scrub_huab242c6c68f4e18b70199053338a4d4f_420340_1200x0_resize_box_3.png 2x,https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-11-grafana-scrub_huab242c6c68f4e18b70199053338a4d4f_420340_1800x0_resize_box_3.png 3x"
  src="https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/2023-10-11-grafana-scrub_huab242c6c68f4e18b70199053338a4d4f_420340_600x0_resize_box_3.png"
  alt="screenshot of a Grafana dashboard showing Prometheus Node Exporter metrics" title="screenshot of a Grafana dashboard showing Prometheus Node Exporter metrics"
  width="600"
  height="226"
  style="

border: 1px solid #000;

"
  
  loading="lazy"></a>



<p>The other maintenance-related setting I changed is to enable automated TRIM:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>zpool <span style="color:#007020">set</span> <span style="color:#bb60d5">autotrim</span><span style="color:#666">=</span>on srv
</span></span></code></pre></div><h4 id="auto-crypto-unlock">Auto Crypto Unlock</h4>
<p>To automatically unlock the encrypted datasets at boot, I’m using a custom <code>unlock.service</code> systemd service file.</p>
<p>My <code>unlock.service</code> constructs the crypto key from two halves: the on-device secret and the remote secret that’s downloaded over HTTPS.</p>
<p>This way, my NAS can boot up automatically, but in an emergency I can remotely stop this mechanism.</p>
<details>
<summary>
My unlock.service
</summary>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-systemd" data-lang="systemd"><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Unit]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Description</span><span style="color:#666">=</span><span style="color:#4070a0">unlock hard drive</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Wants</span><span style="color:#666">=</span><span style="color:#4070a0">network.target</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">After</span><span style="color:#666">=</span><span style="color:#4070a0">systemd-networkd-wait-online.service</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Before</span><span style="color:#666">=</span><span style="color:#4070a0">samba.service</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Service]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Type</span><span style="color:#666">=</span><span style="color:#4070a0">oneshot</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">RemainAfterExit</span><span style="color:#666">=</span><span style="color:#4070a0">yes</span>
</span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># Wait until the host is actually reachable.</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#34;c=0; while [ $c -lt 5 ]; do /bin/ping6 -n -c 1 autounlock.zekjur.net &amp;&amp; break; c=$((c+1)); sleep 1; done&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#34;(echo -n secret &amp;&amp; wget --retry-connrefused -qO - https://autounlock.zekjur.net:8443/nascrypto) | zfs load-key srv/data&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#34;(echo -n secret &amp;&amp; wget --retry-connrefused -qO - https://autounlock.zekjur.net:8443/nascrypto) | zfs load-key srv/backup&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#34;zfs mount srv/data&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#34;zfs mount srv/backup&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Install]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">WantedBy</span><span style="color:#666">=</span><span style="color:#4070a0">multi-user.target</span>
</span></span></code></pre></div></details>
<h3 id="backup">Backup</h3>
<p>For the last 10 years, I have been doing my backups using <code>rsync</code>.</p>
<p>Each machine pushes an incremental backup of its entire root file system (and any mounted file systems that should be backed up, too) to the backup destination (storage2/3).</p>
<p>All the machines I’m backing up run Linux and the <code>ext4</code> file system. I verified that my backup destination file systems support all the features of the backup source file system that I care about, i.e. extended attributes and POSIX ACLs.</p>
<p>The scheduling of backups is done by “<a href="https://github.com/stapelberg/zkj-nas-tools/tree/master/dornroeschen">dornröschen</a>”, a Go program that wakes up the backup sources and destination machines and starts the backup by triggering a command via SSH.</p>
<h4 id="ssh-configuration">SSH configuration</h4>
<p>The backup scheduler establishes an SSH connection to the backup source.</p>
<p>On the backup source, I authorized the scheduler like so, meaning it will run <a href="https://github.com/stapelberg/zkj-nas-tools/blob/master/dornroeschen/backup-remote.pl"><code>/root/backup.pl</code></a> when connecting:</p>
<pre tabindex="0"><code>command=&#34;/root/backup.pl&#34;,no-port-forwarding,no-X11-forwarding ssh-ed25519 AAAAC3Nzainvalidkey backup-scheduler
</code></pre><p>backup.pl runs <code>rsync</code>, which establishes another SSH connection, this time from the backup source to the backup destination.</p>
<p>On the backup destination (storage2/3), I authorize the backup source’s SSH public key to run <a href="https://manpages.debian.org/rrsync.1"><code>rrsync(1)</code></a>
, a script that only permits running <code>rsync</code> in the specified directory:</p>
<pre tabindex="0"><code>command=&#34;/usr/bin/rrsync /srv/backup/server.zekjur.net&#34;,no-port-forwarding,no-X11-forwarding ssh-ed25519 AAAAC3Nzainvalidkey server.zekjur.net
</code></pre><h4 id="signaling-readiness-after-wake-up">Signaling Readiness after Wake-Up</h4>
<p>I found it easiest to signal readiness by starting an empty HTTP server gated on <code>After=unlock.service</code> in systemd:</p>
<details>
<summary><code>/etc/systemd/system/healthz.service</code></summary>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-systemd" data-lang="systemd"><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Unit]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Description</span><span style="color:#666">=</span><span style="color:#4070a0">nginx for /srv health check</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Wants</span><span style="color:#666">=</span><span style="color:#4070a0">network.target</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">After</span><span style="color:#666">=</span><span style="color:#4070a0">unlock.service</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Requires</span><span style="color:#666">=</span><span style="color:#4070a0">unlock.service</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">StartLimitInterval</span><span style="color:#666">=</span><span style="color:#4070a0">0</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Service]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Restart</span><span style="color:#666">=</span><span style="color:#4070a0">always</span>
</span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># https://itectec.com/unixlinux/restarting-systemd-service-on-dependency-failure/</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">/bin/sh -c &#39;systemctl is-active docker.service&#39;</span>
</span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># Stay on the same major version in the hope that nginx never decides to break</span>
</span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># the config file syntax (or features) without doing a major version bump.</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">/usr/bin/docker pull nginx:1</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">-/usr/bin/docker kill nginx-healthz</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">-/usr/bin/docker rm -f nginx-healthz</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/usr/bin/docker run </span>\
</span></span><span style="display:flex;"><span><span style="color:#4070a0">  --name nginx-healthz </span>\
</span></span><span style="display:flex;"><span><span style="color:#4070a0">  --publish 10.0.0.253:8200:80 </span>\
</span></span><span style="display:flex;"><span><span style="color:#4070a0">  --log-driver=journald </span>\
</span></span><span style="display:flex;"><span><span style="color:#4070a0">nginx:1</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Install]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">WantedBy</span><span style="color:#666">=</span><span style="color:#4070a0">multi-user.target</span>
</span></span></code></pre></div></details>
<p>My <a href="https://github.com/stapelberg/zkj-nas-tools/blob/master/wake/wake.go"><code>wake</code></a> program then polls that port and returns once the server is up, i.e. the file system has been unlocked and mounted.</p>
<h4 id="auto-shutdown">Auto Shutdown</h4>
<p>Instead of explicitly triggering a shutdown from the scheduler program, I run “dramaqueen”, which shuts down the machine after 10 minutes, but will be inhibited while a backup is running. Optionally, shutting down can be inhibited while there are active samba sessions.</p>
<details>
<summary><code>/etc/systemd/system/dramaqueen.service</code></summary>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-systemd" data-lang="systemd"><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Unit]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Description</span><span style="color:#666">=</span><span style="color:#4070a0">dramaqueen</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">After</span><span style="color:#666">=</span><span style="color:#4070a0">docker.service</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Requires</span><span style="color:#666">=</span><span style="color:#4070a0">docker.service</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Service]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">Restart</span><span style="color:#666">=</span><span style="color:#4070a0">always</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">StartLimitInterval</span><span style="color:#666">=</span><span style="color:#4070a0">0</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"># Always pull the latest version (bleeding edge).</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">-/usr/bin/docker pull stapelberg/dramaqueen</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">-/usr/bin/docker rm -f dramaqueen</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">/usr/bin/docker create --name dramaqueen stapelberg/dramaqueen</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">/usr/bin/docker cp dramaqueen:/usr/bin/dramaqueen /tmp/</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStartPre</span><span style="color:#666">=</span><span style="color:#4070a0">/usr/bin/docker rm -f dramaqueen</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">ExecStart</span><span style="color:#666">=</span><span style="color:#4070a0">/tmp/dramaqueen -net_command=</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#007020;font-weight:bold">[Install]</span>
</span></span><span style="display:flex;"><span><span style="color:#4070a0">WantedBy</span><span style="color:#666">=</span><span style="color:#4070a0">multi-user.target</span>
</span></span></code></pre></div></details>
<h4 id="enabling-wake-on-lan">Enabling Wake-on-LAN</h4>
<p>Luckily, the network driver of the onboard network card supports WOL by
default. If that’s not the case for your network card, see <a href="https://wiki.archlinux.org/title/Wake-on-LAN">the Arch wiki
Wake-on-LAN article</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I have been running a PC-based few-large-disk Network Storage setup for years at this point, and I am very happy with all the properties of the system. I expect to run a very similar setup for years to come.</p>
<p>The low-tech approach to backups of using rsync has worked well — without changes — for years, and I don’t see rsync going away anytime soon.</p>
<p>The upgrade to all-flash is really nice in terms of random access time (for incremental backups) and to eliminate one of the largest sources of noise from my builds.</p>
<p>ZFS seems to work fine so far and is well-integrated into Ubuntu Server.</p>
<h4 id="related-options">Related Options</h4>
<p>There are solutions for almost everyone’s NAS needs. This build obviously hits my personal sweet spot, but your needs and preferences might be different!</p>
<p>Here are a couple of related solutions:</p>
<ul>
<li>If you would like a more integrated solution, you could take a look at <a href="https://www.heise.de/ratgeber/Einplatinencomputer-Odroid-H3-als-NAS-und-Heimserver-einrichten-7496088.html">the Odroid H3 (Celeron)</a>.</li>
<li>If you’re okay with less compute power, but want more power efficiency, you could use an ARM64-based Single Board Computer.</li>
<li>If you want to buy a commercial solution, buy a device from qnap and fill it with SSD disks.
<ul>
<li>There are even commercial M.2 flash storage devices like the <a href="https://www.jeffgeerling.com/blog/2023/first-look-asustors-new-12-bay-all-m2-nvme-ssd-nas">ASUSTOR Flashstor</a> becoming available! If not for the “off the shelf hardware” goal of my build, this would probably be the most interesting commercial alternative to me.</li>
</ul>
</li>
<li>If you want more compute power, consider a Thin Client (perhaps used) instead of a Single Board Computer.
<ul>
<li><a href="https://www.servethehome.com/">ServeTheHome</a> has a nice series called Project TinyMiniMicro (<a href="https://www.servethehome.com/introducing-project-tinyminimicro-home-lab-revolution/">introduction</a>, <a href="https://www.servethehome.com/tag/tinyminimicro/">blog posts</a>)</li>
<li>If you’re a heise+ subscriber, <a href="https://www.heise.de/ratgeber/Schlank-guenstig-stromsparend-NAS-mit-Thin-Client-im-Eigenbau-7546763.html">they have a (German) article about building a NAS from a thin client</a>.</li>
</ul>
</li>
<li>Very similar to thin clients is the Intel NUC (“Next Unit of Computing”): <a href="https://www.golem.de/news/nuc-12-pro-test-mini-kraftpakete-fuers-buero-und-mediacenter-2303-172992.html">(German) article comparing different NUC 12 devices</a></li>
</ul>


guid: https://michael.stapelberg.ch/posts/2024-01-17-systemd-indefinite-service-restarts/
title: systemd: enable indefinite service restarts
url: https://michael.stapelberg.ch/posts/2024-01-17-systemd-indefinite-service-restarts/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>When a service fails to start up enough times in a row, systemd gives up on it.</p>
<p>On servers, this isn’t what I want — in general it’s helpful for automated
recovery if daemons are restarted indefinitely. As long as you don’t have
circular dependencies between services, all your services will eventually come
up after transient failures, without having to specify dependencies.</p>
<p>This is particularly useful because specifying dependencies on the systemd level
introduces footguns: when interactively stopping individual services, systemd
also stops the dependents. And then you need to remember to restart the
dependent services later, which is easy to forget.</p>
<h2 id="enabling-indefinite-restarts-for-a-service">Enabling indefinite restarts for a service</h2>
<p>To make systemd restart a service indefinitely, I first like to create a drop-in
config file like so:</p>
<pre tabindex="0"><code>cat &gt; /etc/systemd/system/restart-drop-in.conf &lt;&lt;&#39;EOT&#39;
[Unit]
StartLimitIntervalSec=0

[Service]
Restart=always
RestartSec=1s
EOT
</code></pre><p>Then, I can enable the restart behavior for individual services like
<code>prometheus-node-exporter</code>, without having to modify their <code>.service</code> files
(which needs manual effort when updating):</p>
<pre tabindex="0"><code>cd /etc/systemd/system
mkdir prometheus-node-exporter.service.d
cd prometheus-node-exporter.service.d
ln -s ../restart-drop-in.conf
systemctl daemon-reload
</code></pre><h2 id="changing-the-defaults-for-all-services">Changing the defaults for all services</h2>
<p>If most of your services set <code>Restart=always</code> or <code>Restart=on-failure</code>, you can
change the system-wide defaults for <code>RestartSec</code> and <code>StartLimitIntervalSec</code>
like so:</p>
<pre tabindex="0"><code>mkdir /etc/systemd/system.conf.d
cat &gt; /etc/systemd/system.conf.d/restartdefaults.conf &lt;&lt;&#39;EOT&#39;
[Manager]
DefaultRestartSec=1s
DefaultStartLimitIntervalSec=0
EOT
systemctl daemon-reload
</code></pre><h2 id="what-do-the-default-settings-do">What do the default settings do?</h2>
<p>So why do we need to change these settings to begin with?</p>
<p>The default systemd settings (as of systemd 255) are:</p>
<pre tabindex="0"><code>DefaultRestartSec=100ms
DefaultStartLimitIntervalSec=10s
DefaultStartLimitBurst=5
</code></pre><p>This means that services which specify <code>Restart=always</code> are restarted 100ms
after they crash, and if the service crashes more than 5 times in 10 seconds,
systemd does not attempt to restart the service anymore.</p>
<p>It’s easy to see that for a service which takes, say, 100ms to crash, for
example because it can’t bind on its listening IP address, this means:</p>
<table>
<thead>
<tr>
<th>time</th>
<th>event</th>
</tr>
</thead>
<tbody>
<tr>
<td>T+0</td>
<td>first start</td>
</tr>
<tr>
<td>T+100ms</td>
<td>first crash</td>
</tr>
<tr>
<td>T+200ms</td>
<td>second start</td>
</tr>
<tr>
<td>T+300ms</td>
<td>second crash</td>
</tr>
<tr>
<td>T+400ms</td>
<td>third start</td>
</tr>
<tr>
<td>T+500ms</td>
<td>third crash</td>
</tr>
<tr>
<td>T+600ms</td>
<td>fourth start</td>
</tr>
<tr>
<td>T+700ms</td>
<td>fourth crash</td>
</tr>
<tr>
<td>T+800ms</td>
<td>fifth start</td>
</tr>
<tr>
<td>T+900ms</td>
<td>fifth crash within 10s</td>
</tr>
<tr>
<td>T+1s</td>
<td>systemd gives up</td>
</tr>
</tbody>
</table>
<h2 id="why-does-systemd-give-up-by-default">Why does systemd give up by default?</h2>
<p>I’m not sure. If I had to speculate, I would guess the developers wanted to
prevent laptops running out of battery too quickly because one CPU core is
permanently busy just restarting some service that’s crashing in a tight loop.</p>
<p>That same goal could be achieved with a more relaxed <code>DefaultRestartSec=</code> value,
though: With <code>DefaultRestartSec=5s</code>, for example, we would sufficiently space
out these crashes over time.</p>
<p>There is <a href="https://github.com/systemd/systemd/issues/30804">some recent discussion
upstream</a> regarding changing
the default. Let’s see where the discussion goes.</p>


guid: https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/
title: Minimal Linux Bootloader debugging story 🐞
url: https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/
feedurl: https://michael.stapelberg.ch/feed.xml
content_mime_type: text/html
content:
<p>I maintain two builds of the Linux kernel, a <code>linux/arm64</code> build for <a href="https://gokrazy.org">gokrazy,
my Go appliance platform</a>, which started out on the
Raspberry Pi, and then a <code>linux/amd64</code> one for <a href="https://router7.org/">router7</a>,
which runs on PCs.</p>
<p>The update process for both of these builds is entirely automated, meaning new
Linux kernel releases are automatically tested and merged, but recently the
continuous integration testing <a href="https://github.com/rtr7/kernel/pull/434">failed to automatically merge Linux
6․7</a> — this article is about tracking
down the root cause of that failure.</p>
<h2 id="background-info-on-the-bootloader">Background info on the bootloader</h2>
<p>gokrazy started out targeting only the Raspberry Pi, where you configure the
bootloader with a plain text file on a FAT partition, so we did not need to
include our own UEFI/MBR bootloader.</p>
<p>When I ported gokrazy to work on PCs in BIOS mode, I decided against complicated
solutions like GRUB — I really wasn’t looking to maintain a GRUB package. Just
keeping GRUB installations working on my machines is enough work. The fact that
GRUB consists of many different files (modules) that can go out of sync really
does not appeal to me.</p>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">For UEFI, there is <a href="https://en.wikipedia.org/wiki/Systemd-boot">systemd-boot</a>,
which comes as a single-file UEFI program, easy to include. That’s how gokrazy
supports UEFI boot. Unfortunately, the PC Engines apu2c4 does not support UEFI,
so I also needed an MBR solution.</div>
  </div>
</aside>

<p>Instead, I went with Sebastian Plotz’s <a href="https://sebastian-plotz.blogspot.com/2012/07/1.html">Minimal Linux
Bootloader</a> because it fits
entirely into the <a href="https://en.wikipedia.org/wiki/Master_boot_record">Master Boot Record
(MBR)</a> and does not require
any files. In bootloader lingo, this is a stage1-only bootloader. You don’t even
need a C compiler to compile its (Assembly) code. It seemed simple enough to
integrate: just write the bootloader code into the first sector of the gokrazy
disk image; done. The bootloader had its last release in 2012, so no need for
updates or maintenance.</p>
<p>You can’t really implement booting a kernel <strong>and</strong> parsing text configuration
files in <a href="https://en.wikipedia.org/wiki/Master_boot_record#Sector_layout">446
bytes</a> of 16-bit
8086 assembly instructions, so to tell the bootloader where on disk to load the
kernel code and kernel command line from, gokrazy writes the disk offset
(<a href="https://en.wikipedia.org/wiki/Logical_block_addressing">LBA</a>) of <code>vmlinuz</code> and
<code>cmdline.txt</code> to the last bytes of the bootloader code. Because gokrazy
generates the FAT partition, we know there is never any fragmentation, so the
bootloader does not need to understand the FAT file system.</p>
<h2 id="symptom">Symptom</h2>
<p>The symptom was that the <code>rtr7/kernel</code> <a href="https://github.com/rtr7/kernel/pull/434">pull request
#434</a> for updating to Linux 6.7 failed.</p>
<p>My continuous integration tests run in two environments: a physical embedded PC
from <a href="https://pcengines.ch/">PC Engines</a> (apu2c4) in my living room, and a
virtual QEMU PC. Only the QEMU test failed.</p>
<p>On the physical PC Engines apu2c4, the pull request actually passed the boot
test. It would be wrong to draw conclusions like “the issue only affects QEMU”
from this, though, as later attempts to power on the apu2c4 showed the device
boot-looping. I made a mental note that <em>something is different</em> about how the
problem affects the two environments, but both are affected, and decided to
address the failure in QEMU first, then think about the PC Engines failure some
more.</p>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">Later in the investigation I found out that this was because the
physical continuous integration setup didn’t <a href="https://github.com/gokrazy/gokrazy/issues/243">disable kexec
yet</a>, so it wasn’t actually
exercising BIOS boot via the Master Boot Record.</div>
  </div>
</aside>

<p>In QEMU, the output I see is:</p>
<pre tabindex="0"><code>SeaBIOS (version Arch Linux 1.16.3-1-1)

iPXE (http://ipxe.org) 00:03.0 C900 PCI2.10 PnP PMM+06FD3360+06F33360 C900

Booting from Hard Disk...
</code></pre><p>Notably, the kernel doesn’t even seem to start — no “Decompressing linux”
message is printed, the boot just hangs. I tried enabling debug output in
SeaBIOS and <a href="https://github.com/rtr7/router7/issues/83#issuecomment-1890354927">eventually succeeded, but only with an older QEMU
version</a>:</p>
<pre tabindex="0"><code>Booting from Hard Disk...
Booting from 0000:7c00
In resume (status=0)
In 32bit resume
Attempting a hard reboot
</code></pre><p>This doesn’t tell me anything unfortunately.</p>
<p>Okay, so something about introducing Linux 6.7 into my setup breaks MBR boot.</p>
<p>I figured using <a href="https://opensource.com/article/22/11/git-bisect">Git Bisection</a>
should identify the problematic change within a few iterations, so I cloned the
currently working Linux 6.6 source code, applied the router7 config and compiled
it.</p>
<p>To my surprise, even my self-built Linux 6.6 kernel would not boot! 😲</p>
<p>Why does the router7 build work when built inside the Docker container, but not
when built on my Linux installation? I decided to rebase the Docker container
from Debian 10 (buster, from 2019) to Debian 12 (bookworm, from 2023) and that
resulted in a non-booting kernel, too!</p>
<p>We have two triggers: building Linux 6.7 or building older Linux, but in newer
environments.</p>
<h2 id="meta-following-along">Meta: Following Along</h2>
<details>
<summary>(Contains spoilers) Instructions for following along</summary>
<p>First, check out the <code>rtr7/kernel</code> repository and undo <a href="#mitigation">the mitigation</a>:</p>
<pre tabindex="0"><code>% mkdir -p go/src/github.com/rtr7/
% cd go/src/github.com/rtr7/
% git clone --depth=1 https://github.com/rtr7/kernel
% cd kernel
% sed -i &#39;s,CONFIG_KERNEL_ZSTD,#CONFIG_KERNEL_ZSTD,g&#39; cmd/rtr7-build-kernel/config.addendum.txt
% go run ./cmd/rtr7-rebuild-kernel
# takes a few minutes to compile Linux
% ls -l vmlinuz
-rw-r--r-- 1 michael michael 15885312 2024-01-28 16:18 vmlinuz
</code></pre><p>Now, you can either create a new gokrazy instance, replace the kernel and
configure the gokrazy instance to use <code>rtr7/kernel</code>:</p>
<pre tabindex="0"><code>% gok -i mbr new
% gok -i mbr add .
% gok -i mbr edit
# Adjust to contain:
    &#34;KernelPackage&#34;: &#34;github.com/rtr7/kernel&#34;,
    &#34;FirmwarePackage&#34;: &#34;github.com/rtr7/kernel&#34;,
    &#34;EEPROMPackage&#34;: &#34;&#34;,
</code></pre><p>…or you skip these steps and extract <a href="gokrazy-mbr.tar.bz2">my already prepared
config</a> to <code>~/gokrazy/mbr</code>.</p>
<p>Then, build the gokrazy disk image and start it with QEMU:</p>
<pre tabindex="0"><code>% GOARCH=amd64 gok -i mbr overwrite \
  --full /tmp/gokr-boot.img \
  --target_storage_bytes=1258299392
% qemu-system-i386 \
  -nographic \
  -drive file=/tmp/gokr-boot.img,format=raw
</code></pre></details>
<h2 id="updowngrade-versions">Up/Downgrade Versions</h2>
<p>Unlike application programs, the Linux kernel doesn’t depend on shared libraries
at runtime, so the dependency footprint is a little smaller than usual. The most
significant dependencies are the components of the build environment, like the C
compiler or the linker.</p>
<p>So let’s look at the software versions of the known-working (Debian 10)
environment and the smallest change we can make to that (upgrading to Debian
11):</p>
<ul>
<li>Debian 10 (buster) contains gcc-8 (8.3.0-6) and binutils 2.31.1-16.</li>
<li>Debian 11 (bullseye) contains gcc-10 (10.2.1-6) and binutils 2.35.2-2.</li>
</ul>
<p>To figure out if the problem is triggered by GCC, binutils, or something else
entirely, I checked:</p>
<p>Debian 10 (buster) with its <code>gcc-8</code>, but with <code>binutils 2.35</code> from bullseye
still works. (Checked by updating <code>/etc/apt/sources.list</code>, then upgrading only
the <code>binutils</code> package.)</p>
<p>Debian 10 (buster), but with <code>gcc-10</code> and <code>binutils 2.35</code> results in a
non-booting kernel.</p>
<p>So it seems like upgrading from GCC 8 to GCC 10 triggers the issue.</p>
<p>Instead of working with a Docker container and Debian’s packages, you could also
use <a href="https://en.wikipedia.org/wiki/Nix_(package_manager)">Nix</a>. The instructions
aren’t easy, but I <a href="https://github.com/rtr7/router7/issues/83#issuecomment-1885612487">used
<code>nix-shell</code></a>
to quickly try out GCC 8 (works), GCC 9 (works) and GCC 10 (kernel doesn’t boot)
on my machine.</p>
<h2 id="new-hypothesis">New Hypothesis</h2>
<p>To recap, we have two triggers: building Linux 6.7 or building older Linux, but
with GCC 10.</p>
<p>Two theories seemed most plausible to me at this point: Either a change in GCC
10 (possibly enabled by another change in Linux 6.7) is the problem, or the size
of the kernel is the problem.</p>
<p>To verify the file size hypothesis, I padded a known-working <code>vmlinuz</code> file to
the size of a known-broken <code>vmlinuz</code>:</p>
<pre tabindex="0"><code>% ls -l vmlinuz
% dd if=/dev/zero bs=108352 count=1 &gt;&gt; vmlinuz
</code></pre><p>But, even though it had the same file size as the known-broken kernel, the
padded kernel booted!</p>
<p>So I ruled out kernel size as a problem and started researching significant
changes in GCC 10.</p>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">This is an incorrect conclusion! The mistake I made here was that I padded the
kernel on the file level, but the boot loader ignores the file system entirely
and takes the size from the <strong>kernel header</strong>, which I did not update.</div>
  </div>
</aside>

<p>I read that GCC 10 <a href="https://lore.kernel.org/lkml/20200422192113.GG26846@zn.tnic/t/">changed behavior with regards to stack
protection</a>.</p>
<p>Indeed, building the kernel with Debian 11 (bullseye), but with
<code>CONFIG_STACKPROTECTOR=n</code> makes it boot. So, I suspected that our bootloader
does not set up the stack correctly, or similar.</p>
<p>I sent an email to Sebastian Plotz, the author of the Minimal Linux Bootloader,
to ask if he knew about any issues with his bootloader, or if stack protection
seems like a likely issue with his bootloader to him.</p>
<p>To my surprise (it has been over 10 years since he published the bootloader!) he
actually replied: He hadn’t received any problem reports regarding his
bootloader, but didn’t really understand how stack protection would be related.</p>
<h2 id="debugging-with-qemu">Debugging with QEMU</h2>
<p>At this point, we have isolated at least one trigger for the problem, and
exhausted the easy techniques of upgrading/downgrading surrounding software
versions and asking upstream.</p>
<p>It’s time for a <strong>Tooling Level Up</strong>! Without a debugger you can only poke into
the dark, which takes time and doesn’t result in thorough
explanations. Particularly in this case, I think it is very likely that any
source modifications could have introduced subtle issues. So let’s reach for a
debugger!</p>
<p>Luckily, QEMU comes with built-in support for the GDB debugger. Just add the <code>-s -S</code> flags to your QEMU command to make QEMU stop execution (<code>-s</code>) and set up a
GDB stub (<code>-S</code>) listening on <code>localhost:1234</code>.</p>
<p>If you wanted to debug the Linux kernel, you could connect GDB to QEMU right
away, but for debugging a boot loader we need an extra step, because the boot
loader runs in <a href="https://en.wikipedia.org/wiki/Real_mode">Real Mode</a>, but QEMU’s
GDB integration rightfully defaults to the more modern Protected Mode.</p>
<p>When GDB is not configured correctly, it decodes addresses and registers with
the wrong size, which throws off the entire disassembly — compare GDB’s
output with our assembly source:</p>
<pre tabindex="0"><code>(gdb) b *0x7c00
(gdb) c
(gdb) x/20i $pc                         ; [expected (bootloader.asm)]
=&gt; 0x7c00: cli                          ; =&gt; 0x7c00: cli
   0x7c01: xor    %eax,%eax             ;    0x7c01: xor %ax,%ax
   0x7c03: mov    %eax,%ds              ;    0x7c03: mov %ax,%ds
   0x7c05: mov    %eax,%ss              ;    0x7c05: mov %ax,%ss
   0x7c07: mov    $0xb87c00,%esp        ;    0x7c07: mov $0x7c00,%sp
   0x7c0c: adc    %cl,-0x47990440(%esi) ;    0x7c0a: mov $0x1000,%ax
   0x7c12: add    %eax,(%eax)           ;    0x7c0d: mov %ax,%es
   0x7c14: add    %al,(%eax)            ;    0x7c0f: sti
   0x7c16: xor    %ebx,%ebx
</code></pre><p>So we need to ensure we use <code>qemu-system-i386</code> (<code>qemu-system-x86_64</code> prints
<code>Remote 'g' packet reply is too long</code>) and <a href="https://stackoverflow.com/questions/32955887/how-to-disassemble-16-bit-x86-boot-sector-code-in-gdb-with-x-i-pc-it-gets-tr">configure the GDB target
architecture to 16-bit
8086</a>:</p>
<pre tabindex="0"><code>(gdb) set architecture i8086
(gdb) target remote localhost:1234
</code></pre><p>Unfortunately, the above doesn’t actually work in QEMU 2.9 and newer:
<a href="https://gitlab.com/qemu-project/qemu/-/issues/141">https://gitlab.com/qemu-project/qemu/-/issues/141</a>.</p>
<p>On the web, people are working around this bug by <a href="https://gist.github.com/MatanShahar/1441433e19637cf1bb46b1aa38a90815">using a modified <code>target.xml</code>
file</a>. I
tried this, but must have made a mistake — I thought modifying <code>target.xml</code>
didn’t help, but when I wrote this article, I found that it does actually seem
to work. Maybe I didn’t use <code>qemu-system-i386</code> but the <code>x86_64</code> variant or
something like that.</p>
<h2 id="using-an-older-qemu">Using an older QEMU</h2>
<aside class="admonition note">
  <div class="note-container">
    <div class="note-icon" style="width: 20px; height: 20px">
      <svg id="exclamation-icon" width="100%" height="100%" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41421;">
    <path d="M0,0L24,0L24,24L0,24L0,0Z" style="fill:none;"/>
    <g transform="matrix(1.2,0,0,1.2,-2.4,-2.4)">
        <path d="M12,2C6.48,2 2,6.48 2,12C2,17.52 6.48,22 12,22C17.52,22 22,17.52 22,12C22,6.48 17.52,2 12,2ZM13,17L11,17L11,15L13,15L13,17ZM13,13L11,13L11,7L13,7L13,13Z" style="fill-rule:nonzero;"/>
    </g>
</svg>

    </div>
    <div class="admonition-content">As I wrote in the previous paragraph, using an older QEMU might not be necessary
if the <code>target.xml</code> workaround works for you. I decided to leave this section in
because I wanted to showcase the general technique.</div>
  </div>
</aside>

<p>It is typically an exercise in frustration to get older software to compile in newer environments.</p>
<p>It’s much easier to use an older environment to run old software.</p>
<p>By querying <code>packages.debian.org</code>, we can see the <a href="https://packages.debian.org/search?keywords=qemu-system-x86&amp;searchon=names&amp;suite=all&amp;section=all">QEMU versions included in
current and previous Debian
versions</a>.</p>
<p>Unfortunately, the oldest listed version (QEMU 3.1 in Debian 10 (buster)) isn’t
old enough. By querying <code>snapshot.debian.org</code>, we can see that <a href="https://snapshot.debian.org/binary/qemu-system-x86/">Debian 9
(stretch) contained QEMU
2.8</a>.</p>
<p>So let’s run Debian 9 — the easiest way I know is to use Docker:</p>
<pre tabindex="0"><code>% docker run --net=host -v /tmp:/tmp -ti debian:stretch
</code></pre><p>Unfortunately, the <code>debian:stretch</code> Docker container does not work out of the
box anymore, because its <code>/etc/apt/sources.list</code> points to the <code>deb.debian.org</code>
CDN, which only serves current versions and no longer serves <code>stretch</code>.</p>
<p>So we need to update the <code>sources.list</code> file to point to
<code>archive.debian.org</code>. To correctly install QEMU you need both entries, the
<code>debian</code> line and the <code>debian-security</code> line, because the Docker container has
packages from <code>debian-security</code> installed and gets confused when these are
missing from the package list:</p>
<pre tabindex="0"><code>root@650a2157f663:/# cat &gt; /etc/apt/sources.list &lt;&lt;&#39;EOT&#39;
deb http://archive.debian.org/debian/ stretch contrib main non-free
deb http://archive.debian.org/debian-security/ stretch/updates main
EOT
root@650a2157f663:/# apt update
</code></pre><p>Now we can just install QEMU as usual and start it to debug our boot process:</p>
<pre tabindex="0"><code>root@650a2157f663:/# apt install qemu-system-x86
root@650a2157f663:/# qemu-system-i386 \
  -nographic \
  -drive file=/tmp/gokr-boot.img,format=raw \
  -s -S
</code></pre><p>Now let’s start GDB and set a breakpoint on address <code>0x7c00</code>, which is <a href="https://retrocomputing.stackexchange.com/a/21957">the
address to which the BIOS loades the MBR
code</a> and starts execution:</p>
<pre tabindex="0"><code>% gdb
(gdb) set architecture i8086
The target architecture is set to &#34;i8086&#34;.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x0000fff0 in ?? ()
(gdb) break *0x7c00
Breakpoint 1 at 0x7c00
(gdb) continue
Continuing.

Breakpoint 1, 0x00007c00 in ?? ()
(gdb)
</code></pre><h2 id="debug-symbols">Debug symbols</h2>
<p>Okay, so we have GDB attached to QEMU and can step through assembly
instructions. Let’s start debugging!?</p>
<p>Not so fast. There is another Tooling Level Up we need first: debug
symbols. Yes, even for a Minimal Linux Bootloader, which doesn’t use any
libraries or local variables. Having proper names for functions, as well as line
numbers, will be hugely helpful in just a second.</p>
<p>Before debug symbols, I would directly build the bootloader using <code>nasm bootloader.asm</code>, but to end up with a symbol file for GDB, we need to instruct
<code>nasm</code> to generate an ELF file with debug symbols, then use <code>ld</code> to link it and
finally use <code>objcopy</code> to copy the code out of the ELF file again.</p>
<p>After <a href="https://github.com/gokrazy/internal/commit/d29c615f07b8e2632e2178b77d2d3d43dec9d46c">commit
d29c615</a>
in <code>gokrazy/internal/mbr</code>, I have <code>bootloader.elf</code>.</p>
<p>Back in GDB, we can load the symbols using the <code>symbol-file</code> command:</p>
<pre tabindex="0"><code>(gdb) set architecture i8086
The target architecture is set to &#34;i8086&#34;.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x0000fff0 in ?? ()
(gdb) symbol-file bootloader.elf
Reading symbols from bootloader.elf...
(gdb) break *0x7c00
Breakpoint 1 at 0x7c00: file bootloader.asm, line 48.
(gdb) continue
Continuing.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)
</code></pre><h2 id="automation-with-gdbinit">Automation with .gdbinit</h2>
<p>At this point, we need 4 commands each time we start GDB. We can automate these
by writing them to a <code>.gdbinit</code> file:</p>
<pre tabindex="0"><code>% cat &gt; .gdbinit &lt;&lt;&#39;EOT&#39;
set architecture i8086
target remote localhost:1234
symbol-file bootloader.elf
break *0x7c00
EOT

% gdb
The target architecture is set to &#34;i8086&#34;.
0x0000fff0 in ?? ()
Breakpoint 1 at 0x7c00: file bootloader.asm, line 48.
(gdb) 
</code></pre><h2 id="understanding-program-flow">Understanding program flow</h2>
<p>The easiest way to understand program flow seems to be to step through the program.</p>
<p>But Minimal Linux Bootloader (MLB) contains loops that run through thousands of
iterations. You can’t use gdb’s <code>stepi</code> command with that.</p>
<p>Because MLB only contains a few functions, I eventually realized that placing a
breakpoint on each function would be the quickest way to understand the
high-level program flow:</p>
<pre tabindex="0"><code>(gdb) b read_kernel_setup
Breakpoint 2 at 0x7c38: file bootloader.asm, line 75.
(gdb) b check_version
Breakpoint 3 at 0x7c56: file bootloader.asm, line 88.
(gdb) b read_protected_mode_kernel
Breakpoint 4 at 0x7c8f: file bootloader.asm, line 105.
(gdb) b read_protected_mode_kernel_2
Breakpoint 5 at 0x7cd6: file bootloader.asm, line 126.
(gdb) b run_kernel
Breakpoint 6 at 0x7cff: file bootloader.asm, line 142.
(gdb) b error
Breakpoint 7 at 0x7d51: file bootloader.asm, line 190.
(gdb) b reboot
Breakpoint 8 at 0x7d62: file bootloader.asm, line 204.
</code></pre><p>With the working kernel, we get the following transcript:</p>
<pre tabindex="0"><code>(gdb)
Continuing.

Breakpoint 2, read_kernel_setup () at bootloader.asm:75
75		xor	eax, eax
(gdb)
Continuing.

Breakpoint 3, check_version () at bootloader.asm:88
88		cmp	word [es:0x206], 0x204		; we need protocol version &gt;= 2.04
(gdb)
Continuing.

Breakpoint 4, read_protected_mode_kernel () at bootloader.asm:105
105		mov	edx, [es:0x1f4]			; edx stores the number of bytes to load
(gdb)
Continuing.

Breakpoint 5, read_protected_mode_kernel_2 () at bootloader.asm:126
126		mov	eax, edx
(gdb)
Continuing.

Breakpoint 6, run_kernel () at bootloader.asm:142
142		cli
(gdb)
</code></pre><p>With the non-booting kernel, we get:</p>
<pre tabindex="0"><code>(gdb) c
Continuing.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)
Continuing.

Breakpoint 2, read_kernel_setup () at bootloader.asm:75
75		xor	eax, eax
(gdb)
Continuing.

Breakpoint 3, check_version () at bootloader.asm:88
88		cmp	word [es:0x206], 0x204		; we need protocol version &gt;= 2.04
(gdb)
Continuing.

Breakpoint 4, read_protected_mode_kernel () at bootloader.asm:105
105		mov	edx, [es:0x1f4]			; edx stores the number of bytes to load
(gdb)
Continuing.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)
</code></pre><p>Okay! Now we see that the bootloader starts loading the kernel from disk into
RAM, but doesn’t actually get far enough to call <code>run_kernel</code>, meaning the
problem isn’t with stack protection, with loading a working command line or with
anything <em>inside</em> the Linux kernel.</p>
<p>This lets us rule out a large part of the problem space. We now know that we can
focus entirely on the bootloader and why it cannot load the Linux kernel into
memory.</p>
<p>Let’s take a closer look…</p>
<h2 id="wait-this-isnt-gdb">Wait, this isn’t GDB!</h2>
<p>In the example above, using breakpoints was sufficient to narrow down the problem.</p>
<p>You might think we used GDB, and it looked like this:</p>















<a href="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-text.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-text_hufb947737200a71a40def7b55daedda4a_651685_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-text_hufb947737200a71a40def7b55daedda4a_651685_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-text_hufb947737200a71a40def7b55daedda4a_651685_600x0_resize_q75_box.jpg"
  
  width="600"
  height="589"
  style="

border: 0;

"
  
  loading="lazy"></a>



<p>But that’s not GDB! It’s an easy mistake to make. After all, GDB starts up with
just a text prompt, and as you can see from the example above, we can just enter
text and achieve a good result.</p>
<p>To see the real GDB, you need to start it up fully, meaning including its user
interface.</p>
<p>You can either use GDB’s text user interface (TUI), or a graphical user
interface for gdb, such as the one available in Emacs.</p>
<h3 id="the-gdb-text-mode-user-interface-tui">The GDB text-mode user interface (TUI)</h3>
<p>You’re already familiar with the <code>architecture</code>, <code>target</code> and <code>breakpoint</code>
commands from above. To also set up the text-mode user interface, we run a few
<code>layout</code> commands:</p>
<pre tabindex="0"><code>(gdb) set architecture i8086
(gdb) target remote localhost:1234
(gdb) symbol-file bootloader.elf
(gdb) layout split
(gdb) layout src
(gdb) layout regs
(gdb) break *0x7c00
(gdb) continue
</code></pre><p>The <code>layout split</code> command loads the text-mode user interface and splits the
screen into a register window, disassembly window and command window.</p>
<p>With <code>layout src</code> we disregard the disassembly window in favor of a source
listing window. Both are in assembly language in our case, but the source
listing contains comments as well.</p>
<p>The <code>layout src</code> command also got rid of the register window, which we’ll get
back using <code>layout regs</code>. I’m not sure if there’s an easier way.</p>
<p>The result looks like this:</p>















<a href="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-featured.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-featured_hucd7e1463fcc59a2458cbdaeb4c4f836a_674116_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-featured_hucd7e1463fcc59a2458cbdaeb4c4f836a_674116_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-featured_hucd7e1463fcc59a2458cbdaeb4c4f836a_674116_600x0_resize_q75_box.jpg"
  
  width="600"
  height="583"
  style="

border: 0;

"
  
  loading="lazy"></a>



<p>The source window will highlight the next line of code that will be executed. On
the left, the <code>B+</code> marker indicates an enabled breakpoint, which will become
helpful with multiple breakpoints. Whenever a register value changes, the
register and its new value will be highlighted.</p>
<p>The up and down arrow keys scroll the source window.</p>
<p>Use <code>C-x o</code> to switch between the windows.</p>
<p>If you’re familiar with Emacs, you’ll recognize the keyboard shortcut. But as an
Emacs user, you might prefer the GDB Emacs user interface:</p>
<h3 id="the-gdb-emacs-user-interface-m-x-gdb">The GDB Emacs user interface (M-x gdb)</h3>
<p>This is <code>M-x gdb</code> with <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/GDB-User-Interface-Layout.html"><code>gdb-many-windows</code>
enabled</a>:</p>















<a href="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-emacs.jpg"><img
  srcset="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-emacs_huea0ff23d44f02f4aa1fe1640e7e9fb8a_801503_1200x0_resize_q75_box.jpg 2x,https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-emacs_huea0ff23d44f02f4aa1fe1640e7e9fb8a_801503_1800x0_resize_q75_box.jpg 3x"
  src="https://michael.stapelberg.ch/posts/2024-02-11-minimal-linux-bootloader-debugging-story/2024-01-21-gdb-emacs_huea0ff23d44f02f4aa1fe1640e7e9fb8a_801503_600x0_resize_q75_box.jpg"
  
  width="600"
  height="584"
  style="

border: 0;

"
  
  loading="lazy"></a>



<h2 id="debugging-the-failing-loop">Debugging the failing loop</h2>
<p>Let’s take a look at the loop that we know the bootloader is entering, but not
leaving (neither <code>read_protected_mode_kernel_2</code> nor <code>run_kernel</code> are ever called):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-asm" data-lang="asm"><span style="display:flex;"><span><span style="color:#002070;font-weight:bold">read_protected_mode_kernel:</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">edx</span>, [<span style="color:#60add5">es</span>:<span style="color:#40a070">0x1f4</span>]              <span style="color:#60a0b0;font-style:italic">; edx stores the number of bytes to load
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">shl</span>  <span style="color:#60add5">edx</span>, <span style="color:#40a070">4</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#002070;font-weight:bold">.loop:</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">cmp</span>  <span style="color:#60add5">edx</span>, <span style="color:#40a070">0</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">je</span>   <span style="color:#60add5">run_kernel</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">cmp</span>  <span style="color:#60add5">edx</span>, <span style="color:#40a070">0xfe00</span>                  <span style="color:#60a0b0;font-style:italic">; less than 127*512 bytes remaining?
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">jb</span>   <span style="color:#60add5">read_protected_mode_kernel_2</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">eax</span>, <span style="color:#40a070">0x7f</span>                    <span style="color:#60a0b0;font-style:italic">; load 127 sectors (maximum)
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">xor</span>  <span style="color:#60add5">bx</span>, <span style="color:#60add5">bx</span>                       <span style="color:#60a0b0;font-style:italic">; no offset
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">cx</span>, <span style="color:#40a070">0x2000</span>                   <span style="color:#60a0b0;font-style:italic">; load temporary to 0x20000
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">esi</span>, <span style="color:#60add5">current_lba</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">call</span> <span style="color:#60add5">read_from_hdd</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">cx</span>, <span style="color:#40a070">0x7f00</span>                   <span style="color:#60a0b0;font-style:italic">; move 65024 bytes (127*512 byte)
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">call</span> <span style="color:#60add5">do_move</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">sub</span>  <span style="color:#60add5">edx</span>, <span style="color:#40a070">0xfe00</span>                  <span style="color:#60a0b0;font-style:italic">; update the number of bytes to load
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">add</span>  <span style="color:#60add5">word</span> [<span style="color:#60add5">gdt.dest</span>], <span style="color:#40a070">0xfe00</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">adc</span>  <span style="color:#60add5">byte</span> [<span style="color:#60add5">gdt.dest</span><span style="">+</span><span style="color:#40a070">2</span>], <span style="color:#40a070">0</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">jmp</span>  <span style="color:#60add5">short</span> <span style="color:#60add5">read_protected_mode_kernel.loop</span>
</span></span></code></pre></div><p>The comments explain that the code loads chunks of FE00h == 65024 (127*512)
bytes at a time.</p>
<p>Loading means calling <code>read_from_hdd</code>, then <code>do_move</code>. Let’s take a look at <code>do_move</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-asm" data-lang="asm"><span style="display:flex;"><span><span style="color:#002070;font-weight:bold">do_move:</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">push</span> <span style="color:#60add5">edx</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">push</span> <span style="color:#60add5">es</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">xor</span>  <span style="color:#60add5">ax</span>, <span style="color:#60add5">ax</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">es</span>, <span style="color:#60add5">ax</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">ah</span>, <span style="color:#40a070">0x87</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">mov</span>  <span style="color:#60add5">si</span>, <span style="color:#60add5">gdt</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">int</span>  <span style="color:#40a070">0x15</span>     <span style="color:#60a0b0;font-style:italic">; line 182
</span></span></span><span style="display:flex;"><span><span style="color:#60a0b0;font-style:italic"></span>    <span style="color:#06287e">jc</span>   <span style="color:#60add5">error</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">pop</span>  <span style="color:#60add5">es</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">pop</span>  <span style="color:#60add5">edx</span>
</span></span><span style="display:flex;"><span>    <span style="color:#06287e">ret</span>
</span></span></code></pre></div><p><code>int 0x15</code> is a call to the BIOS Service Interrupt, which will dispatch the call
based on <code>AH == 87H</code> to the <a href="http://www.techhelpmanual.com/222-int_15h_87h__move_memory_block.html">Move Memory Block
(techhelpmanual.com)</a>
function.</p>
<p>This function moves the specified amount of memory (65024 bytes in our case)
from source/destination addresses specified in a Global Descriptor Table (GDT)
record.</p>
<p>We can use GDB to show the addresses of each of <code>do_move</code>’s memory move calls by
telling it to stop at line 182 (the <code>int 0x15</code> instruction) and print the GDT
record’s destination descriptor:</p>
<pre tabindex="0"><code>(gdb) break 182
Breakpoint 2 at 0x7d49: file bootloader.asm, line 176.

(gdb) command 2
Type commands for breakpoint(s) 2, one per line.
End with a line saying just &#34;end&#34;.
&gt;x/8bx gdt+24
&gt;end

(gdb) continue
Continuing.

Breakpoint 1, ?? () at bootloader.asm:48
42		cli

(gdb)
Continuing.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x00	0x10	0x93	0x00	0x00

(gdb)
Continuing.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0xfe	0x10	0x93	0x00	0x00

(gdb)
</code></pre><p>The destination address is stored in byte <code>2..4</code>. Remember to read these little
endian entries “back to front”.</p>
<ul>
<li>
<p>Address #1 is <code>0x100000</code>.</p>
</li>
<li>
<p>Address #2 is <code>0x10fe00</code>.</p>
</li>
</ul>
<p>If we press Return long enough, we eventually end up here:</p>
<pre tabindex="0"><code>Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x1e	0xff	0x93	0x00	0x00
(gdb)
Continuing.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x1c	0x00	0x93	0x00	0x00

(gdb)
Continuing.

Breakpoint 1, ?? () at bootloader.asm:48
42		cli
(gdb)

Program received signal SIGTRAP, Trace/breakpoint trap.
0x000079b0 in ?? ()
(gdb)
</code></pre><p>Now that execution left the bootloader, let’s take a look at the last <code>do_move</code>
call parameters: We notice that the destination address overflowed its 24 byte
data type:</p>
<ul>
<li>Address #y is <code>0xff1e00</code></li>
<li>Address #z is <code>0x001c00</code></li>
</ul>
<h2 id="root-cause">Root cause</h2>
<p>At this point I reached out to Sebastian again to ask him if there was an
(undocumented) fundamental architectural limit to his Minimal Linux Bootloader —
with 24 bit addresses, you can address at most 16 MB of memory.</p>
<p>He replied explaining that he didn’t know of this limit either! He then linked
to <a href="http://www.techhelpmanual.com/222-int_15h_87h__move_memory_block.html">Move Memory Block
(techhelpmanual.com)</a>
as proof for the 24 bit limit.</p>
<h3 id="speculation">Speculation</h3>
<p>So, is it impossible to load larger kernels into memory from Real Mode? I’m not
sure.</p>
<p>The current bootloader code prepares a GDT in which addresses are 24 bits long
at most. But note that the techhelpmanual.com documentation that Sebastian
referenced is apparently for the <a href="https://en.wikipedia.org/wiki/Intel_80286">Intel
286</a> (a 16 bit CPU), and some of the
GDT bytes are declared reserved.</p>
<p>Today’s CPUs are <a href="https://en.wikipedia.org/wiki/I386">Intel 386</a>-compatible (a
32 bit CPU), which seems to use one of the formerly reserved bytes to represent
bit <code>24..31</code> of the address, meaning we might be able to pass 32 bit addresses
to BIOS functions in a GDT after all!</p>
<p>I wasn’t able to find clear authoritative documentation on the Move Memory Block
API on 386+, or whether BIOS functions in general are just expected to work with 32 bit addresses.</p>
<p>But Microsoft’s 1989 <a href="https://github.com/MikeyG/himem/blob/e041532abee44d663067dc6c2b782e459081fa14/oemsrc/xm386.asm#L12">HIMEM.SYS source contains a
struct</a>
that documents this 32-bit descriptor usage. A more modern reference is this
<a href="https://sys.cs.fau.de/extern/lehre/ws23/bs/uebung/seminar/boot.pdf">Operating Systems Class from FAU
2023</a> (page
71/72).</p>
<p>Hence I’m <em>thinking</em> that most BIOS implementations should actually support 32
bit addresses for their Move Memory Block implementation — provided you fill the
descriptor accordingly.</p>
<p>If that doesn’t work out, there’s also <a href="https://www.os2museum.com/wp/a-brief-history-of-unreal-mode/">“Unreal
Mode”</a>, which
allows using up to 4 GB in Real Mode, but is a change that is a lot more
complicated. See also <a href="https://blogsystem5.substack.com/p/beyond-the-1-mb-barrier-in-dos">Julio Merino’s “Beyond the 1 MB barrier in DOS”
post</a> to get
an idea of the amount of code needed.</p>
<h3 id="update-a-fix">Update: a fix!</h3>
<p><a href="https://lobste.rs/s/kaj3c2/minimal_linux_bootloader_debugging#c_ybraf4">Lobsters reader abbeyj pointed
out</a>
that the following code change should fix the truncation and result in a GDT
with all address bits in the right place:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-diff" data-lang="diff"><span style="display:flex;"><span><span style="color:#a00000">--- i/mbr/bootloader.asm
</span></span></span><span style="display:flex;"><span><span style="color:#a00000"></span><span style="color:#00a000">+++ w/mbr/bootloader.asm
</span></span></span><span style="display:flex;"><span><span style="color:#00a000"></span><span style="color:#800080;font-weight:bold">@@ -119,6 +119,7 @@ read_protected_mode_kernel:
</span></span></span><span style="display:flex;"><span><span style="color:#800080;font-weight:bold"></span> 	sub	edx, 0xfe00			; update the number of bytes to load
</span></span><span style="display:flex;"><span> 	add	word [gdt.dest], 0xfe00
</span></span><span style="display:flex;"><span> 	adc	byte [gdt.dest+2], 0
</span></span><span style="display:flex;"><span><span style="color:#00a000">+	adc	byte [gdt.dest+5], 0
</span></span></span><span style="display:flex;"><span><span style="color:#00a000"></span> 	jmp	short read_protected_mode_kernel.loop
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> read_protected_mode_kernel_2:
</span></span></code></pre></div><p>…and indeed, in my first test this seems to fix the problem! It’ll take me a
little while to clean this up and submit it. You can follow <a href="https://github.com/gokrazy/gokrazy/issues/248">gokrazy issue
#248</a> if you’re interested.</p>
<h2 id="bonus-reading-bios-source">Bonus: reading BIOS source</h2>
<p>There are actually a couple of BIOS implementations that we can look into to get
a better understanding of how Move Memory Block works.</p>
<p>We can look at <a href="https://en.wikipedia.org/wiki/DOSBox">DOSBox</a>, an open source
DOS emulator. Its <a href="https://sourceforge.net/p/dosbox/code-0/HEAD/tree/dosbox/branches/0_74_3/src/ints/bios.cpp#l663">Move Memory Block
implementation</a>
does seem to support 32 bit addresses:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f0f0f0;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-c" data-lang="c"><span style="display:flex;"><span>PhysPt dest	<span style="color:#666">=</span> (<span style="color:#06287e">mem_readd</span>(data<span style="color:#666">+</span><span style="color:#40a070">0x1A</span>) <span style="color:#666">&amp;</span> <span style="color:#40a070">0x00FFFFFF</span>) <span style="color:#666">+</span>
</span></span><span style="display:flex;"><span>              (<span style="color:#06287e">mem_readb</span>(data<span style="color:#666">+</span><span style="color:#40a070">0x1E</span>)<span style="color:#666">&lt;&lt;</span><span style="color:#40a070">24</span>);
</span></span></code></pre></div><p>Another implementation is <a href="https://www.seabios.org/SeaBIOS">SeaBIOS</a>. Contrary
to DOSBox, SeaBIOS is not just used in emulation: The PC Engines apu uses
coreboot with SeaBIOS. QEMU also uses SeaBIOS.</p>
<p><a href="https://github.com/qemu/seabios/blob/ea1b7a0733906b8425d948ae94fba63c32b1d425/src/system.c#L72">The SeaBIOS <code>handle_1587</code> source
code</a>
is a little harder to follow, because it requires knowledge of Real Mode
assembly. The way I read it, SeaBIOS doesn’t truncate or otherwise modify the
descriptors and just passes them to the CPU. On 386 or newer, 32 bit addresses
should work.</p>
<h2 id="mitigation">Mitigation</h2>
<p>While it’s great to understand the limitation we’re running into, I wanted to
unblock the pull request as quickly as possible, so I needed a quick mitigation
instead of investigating if <a href="#speculation">my speculation</a> can be developed into
a proper fix.</p>
<p>When I started router7, we didn’t support loadable kernel modules, so everything
had to be compiled into the kernel. We now do support loadable kernel modules,
so I could have moved functionality into modules.</p>
<p>Instead, I found an even easier quick fix: <a href="https://github.com/rtr7/kernel/commit/304a623297fe3b7ae303811ac097c01fcca901e0">switching from gzip to zstd
compression</a>. This
saved about 1.8 MB and will buy us some time to implement a proper fix while
unblocking automated new Linux kernel version merges.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I wanted to share this debugging story because it shows a couple of interesting lessons:</p>
<ol>
<li>
<p>Being able to run older versions of various parts of your software stack is a
very valuable debugging tool. It helped us isolate a trigger for the bug
(using an older GCC) and it helped us set up a debugging environment (using
an older QEMU).</p>
</li>
<li>
<p>Setting up a debugger can be annoying (symbol files, learning the UI) but
it’s <em>so worth it</em>.</p>
</li>
<li>
<p>Be on the lookout for wrong turns during debugging. Write down every
conclusion and challenge it.</p>
</li>
<li>
<p>The BIOS can seem mysterious and “too low level” but there are many blog
posts, lectures and tutorials. You can also just read open-source BIOS code
to understand it much better.</p>
</li>
</ol>
<p>Enjoy poking at your BIOS!</p>
<h2 id="appendix-resources">Appendix: Resources</h2>
<p>I found the following resources helpful:</p>
<ul>
<li><a href="https://0xax.gitbook.io/linux-insides/summary/booting/linux-bootstrap-1">linux-insides: From bootloader to kernel</a></li>
<li><a href="https://www.pcjs.org/documents/books/mspl13/msdos/encyclopedia/section2/">The MS-DOS Encyclopedia</a></li>
<li><a href="https://www.youtube.com/watch?v=0q6Ujn_zNH8">Ben Eater: A simple BIOS for my breadboard computer (22 minutes)</a></li>
</ul>


