{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "※ hackd",
  "description": "hackd - security & risk",
  "home_page_url": "https://hackd.net/",
  "feed_url": "https://hackd.net/feed.json",
  "language": "en-us",
  "authors": [
    {
      "name": "roguesys"
    }
  ],
  "items": [
    {
      "id": "https://hackd.net/posts/offensive-security-is-a-reality-test/",
      "url": "https://hackd.net/posts/offensive-security-is-a-reality-test/",
      "external_url": "https://andywgrant.substack.com/p/offensive-security-is-a-reality-test",
      "title": "Offensive security is a reality test",
      "content_html": "<p>Distilling what a Red Team does can be difficult at times, because frequently\npeople focus on the tactical findings: this vulnerability, that\nmisconfiguration, some control that didn't work or doesn't exist. Those are all\nvaluable and important aspects of the work, no doubt, and they're the common\nartifacts of operations. But they're not the essence.</p>\n<p>I like Andy's characterization of red teaming as a truth-finding function, and\nit's very close to how I've been describing the work as well. Taking a\ncontrarian position to a given claim (\"Our data is secure.\") and then working to\nvalidate that hypothesis, to find the truth.</p>",
      "date_published": "2026-08-05T18:11:29.239Z",
      "date_modified": "2026-08-05T18:11:29.239Z",
      "tags": [
        "quote",
        "offensive security",
        "red team"
      ]
    },
    {
      "id": "https://hackd.net/posts/prompt-injection-as-role-confusion/",
      "url": "https://hackd.net/posts/prompt-injection-as-role-confusion/",
      "external_url": "https://role-confusion.github.io/",
      "title": "Prompt injection as role confusion",
      "content_html": "<p>It seems intuitively true that it is not possible to secure a system with an\napproach that's enmeshed with the input mechanism through which attacks also\narrive; we've done much better using out-of-band controls to avoid commingling\ncode and data. Still, it's great to see this research land and confidently show\nthis:</p>\n<blockquote>\n<p>Role tags were a formatting trick that became the security architecture and\nthe cognitive scaffolding of modern LLMs. We've shown that this architecture\ndoesn't survive into the model's actual representations, and that such role\nconfusion is linked to prompt injection.</p>\n<p>Unless LLMs achieve genuine role perception, we think injection defense will\nremain a perpetual whack-a-mole game. And the continuous nature of role\nboundaries opens the threat of injections designed to subtly shift LLM states\nthrough seemingly innocuous text, legally and at scale.</p>\n</blockquote>",
      "date_published": "2026-06-23T03:52:27.233Z",
      "date_modified": "2026-06-23T03:52:27.233Z",
      "tags": [
        "ai",
        "llm",
        "security"
      ]
    },
    {
      "id": "https://hackd.net/posts/we-are-living-in-pinocchios-world/",
      "url": "https://hackd.net/posts/we-are-living-in-pinocchios-world/",
      "external_url": "https://om.co/2026/05/25/we-are-living-in-pinocchios-world/",
      "title": "We are living in Pinocchio's world",
      "content_html": "<blockquote>\n<p>The grifters and the hucksters and the influencers selling impossible things\nsucceed because audiences reward certainty and punish doubt. They honor\nconfidence and resist complication. A clean story about a genius who will fix\neverything travels faster than a difficult story about tradeoffs.</p>\n</blockquote>\n<p>Skepticism is always healthy. Mostly everything has trade-offs, apparent or not,\nso you have to keep going until you can see their shape.</p>",
      "date_published": "2026-06-03T17:06:56.518Z",
      "date_modified": "2026-06-03T17:06:56.518Z",
      "tags": [
        "quote"
      ]
    },
    {
      "id": "https://hackd.net/posts/uv-lockfiles-and-malware-on-pypi/",
      "url": "https://hackd.net/posts/uv-lockfiles-and-malware-on-pypi/",
      "title": "uv lock files and malware on PyPI",
      "content_html": "<p>I'm pretty sure that saying \"software supply chain security is important\" won't\nwin anyone a prize, yet we see quite an alarming ramp-up in malicious campaigns\nthat take advantage of various gaps in the space, despite all the industry\nattention. Some problems take time to fix, but there's a lot more we could be\ndoing today that isn't for lack of technology, but organizational friction.\nAnyway, we're not here to discuss that right now.</p>\n<p>The <a href=\"https://www.wiz.io/blog/threes-a-crowd-teampcp-trojanizes-litellm-in-continuation-of-campaign%3E\">LiteLLM compromise</a> was interesting to watch, especially the\nimpact on downstream consumers. Essentially, everyone that installed or happened\nto update the <code>LiteLLM</code> package (or something that depended on it) got\ncompromised, as the newest (malicious) version was pulled in their environment.</p>\n<p>One of the ways to reduce this kind of thing from happening is to use lock files\nin the package managers that support them. Lock files are a record of all\ndependencies for a given package, including versions and cryptographic hashes.\nUpdating the lock file happens intentionally, during development, but deploying\nthe package in production uses the lock file as-is, installing known-good\nversions as specified. This also has some other non-security related properties,\nlike making sure the versions of packages you're deploying won't magically break\nthings because of upstream changes. So you do your dev work, add deps as needed,\nupdate the lock file, and ship that. It's also best practice to pin your\ndependencies to a specific version (like <code>==X.Y.Z</code>) instead of using softer\nconstraints that may allow for updates (e.g., <code>>=X.Y.0</code> installing <code>X.Y.23</code>).\nThis pinning usually happens in a configuration file adjacent to the lock file,\nlike <code>pyproject.toml</code> or <code>Cargo.toml</code>.</p>\n<h2>The surprise</h2>\n<p>Let's talk concretely about Python:</p>\n<ul>\n<li>Define dependencies (and other stuff) in <code>pyproject.toml</code>.</li>\n<li>Use <code>uv</code> because it actually lets you create and use a lock file, <code>uv.lock</code>\n(and it's fast).</li>\n<li>As needed, run <code>uv sync</code> and record all package versions, hashes, and download\nURLs in the lock file.</li>\n<li>When building for prod, use <code>uv sync --frozen</code> which will <strong>only</strong> use what's\nin the lock file, installing the same package versions regardless of upstream\nchanges. This uses the URLs directly, and won't check for any newer versions\nof packages, even if the constraints would allow for updates.</li>\n</ul>\n<p>PyPI is the free package hosting and distribution solution for Python, and it's\na wonderful service to be grateful for. It's where <code>uv</code> (and <code>pip</code>) look for by\ndefault for any Python dependency. So usually <code>uv.lock</code> will contain quite a few\nentries for things hosted there, and download URLs like\n<code>https://files.pythonhosted.org/packages/12/34/big-hash/my_pkg-X.Y.Z-py3-none-any.whl</code>\nwhich provide the location of <code>my-pkg</code> at version <code>X.Y.Z</code>.</p>\n<p>But what if version <code>X.Y.Z</code> is compromised and malicious? Well, PyPI or a\nmaintainer can delete it, which removes the package from the index, and it won't\nbe installable anymore.</p>\n<p>Well, not quite.</p>\n<p>The package does get removed from the index, true, but <strong>the URL remains active\nas a result of how storage works</strong>, and the PyPI CDN will happily continue\nserving it. Which means subsequent <code>uv sync --frozen</code> invocations will install\nthat malicious package, since this form doesn't consult the registry anymore,\nand just directly fetches the URL.</p>\n<p>So, if during the window of time that an upstream package is compromised, one or\nmore lock files get updated to point to that malicious URL, the bad package will\nstill get installed. There can be a delay here until a defender might see these\nlock files deployed, especially given the nuance of unpinned versions and other\nthings, so having an effective inventory of all installed packages to query is\nvery important to properly address this. Package aging (delaying updates) and\nother mitigations are also part of the defensive story here, but ultimately it's\nworth knowing what gets installed and where.</p>\n<p>This isn't really the fault of PyPI or <code>uv</code>, just how these two things\ninterplay. <code>uv</code> is <a href=\"https://github.com/astral-sh/uv/issues/18781\">thinking about the problem</a> and how to fix it.</p>",
      "summary": "A potentially surprising interplay between these two systems means you might get compromised by deleted packages.",
      "date_published": "2026-04-07T15:43:05.118Z",
      "date_modified": "2026-04-07T15:43:05.118Z",
      "tags": [
        "security",
        "python",
        "supply chain security"
      ]
    },
    {
      "id": "https://hackd.net/posts/choice-responsibility/",
      "url": "https://hackd.net/posts/choice-responsibility/",
      "title": "Choices and responsibility",
      "content_html": "<p>Every time new technology comes out and becomes widely available, there is a\ndifficult balance to be found between the exuberance of early adopters—keen to\ndrive broad usage and land early wins unlocked by the new thing—<sup><a href=\"#user-content-fn-em\" id=\"user-content-user-content-fnref-em\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">1</a></sup>and those\nwho would seek to better understand the downsides before going too deep. There\nare essential (and obvious) risks none disagree on, but these are few. The real\nchallenge is in negotiating the middle-ground, all the second+ order effects.\nIt's fair that neither side prevails unchallenged<sup><a href=\"#user-content-fn-balance\" id=\"user-content-user-content-fnref-balance\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">2</a></sup>.</p>\n<p>My primary perspective comes from doing Security Engineering work across\ncompanies of different sizes (and cultures), some defense but mostly offense. On\neither side though, I've largely thought about what the new technology amplifies\nin terms of risk. Today, of course, it's LLMs, but the problem space is fractal:\nwe run into variants<sup><a href=\"#user-content-fn-inject\" id=\"user-content-user-content-fnref-inject\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">3</a></sup> of the same meta issues all the time in security,\nbecause that's the nature of our work. Something shiny, often useful, almost\nnever designed with safety<sup><a href=\"#user-content-fn-safety\" id=\"user-content-user-content-fnref-safety\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">4</a></sup> as a primary property.</p>\n<p>Yet, in accepting that we cannot wait for good-enough guardrails around new\ntechnology within certain time constraints, we also <strong>do not get to abdicate our\nresponsibility</strong> for the choice of using such tech. We might not be aware of all\nthe specific second+ order effects, but we can be sure they exist<sup><a href=\"#user-content-fn-macro\" id=\"user-content-user-content-fnref-macro\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">5</a></sup>.</p>\n<p>So it is with LLMs and all they've recently unlocked in terms of working with\nsoftware and systems (e.g., code assistance, vulnerability discovery, semi- or\nfully- autonomous personal agents). Broadly, we've been able to convert A LOT of\nideas into running code, and got some pretty funny and clever things out, too.\nQuite fast, generally. Yet, all of this software remains the responsibility of\nwhatever human actor is ultimately at the top of the pyramid, and we should\nneither pretend otherwise, nor enable unaccountability in this regard.</p>\n<p>There are situations where and simply using whatever the LLM generated for code\nis fine without paying it too much attention: prototypes, one-off scripts, even\ntools that would only ever impact that human if there was a problem (i.e., skin\nin the game), etc.</p>\n<p>Yet for code that's to be shared with other people it remains the human's\nresponsibility to review it, ensure its quality, and do so both in terms of\nrespect, and empathy, before sharing it in the first place. That human is still\nresponsible.</p>\n<hr>\n<p>I really enjoyed reading (and recommend) both Russ Cox's\n<a href=\"https://groups.google.com/g/golang-dev/c/4Li4Ovd_ehE/m/8L9s_jq4BAAJ\">post in <code>golang-dev@</code></a>\non suggestions for how to approach aspects of this issue in that project, and\nthe <a href=\"https://rfd.shared.oxide.computer/rfd/0576\">Oxide RFD (576)</a> he mentions.</p>\n<section data-footnotes class=\"footnotes\"><h2 class=\"sr-only\" id=\"user-content-footnote-label\">Footnotes</h2>\n<ol>\n<li id=\"user-content-user-content-fn-em\">\n<p>I happen to know how to use em-dashes; this article is 100% organic. <a href=\"#user-content-fnref-em\" data-footnote-backref=\"\" aria-label=\"Back to reference 1\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-balance\">\n<p>Even this concession is uncomfortable for some cultures, which would much\nrather build up a high degree of confidence/containment before allowing\nanything. This moves into a different discussion on risk, and while that\nconversation is interesting and worth iterating on, it is not how things\ncurrently work in practice. <a href=\"#user-content-fnref-balance\" data-footnote-backref=\"\" aria-label=\"Back to reference 2\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-inject\">\n<p>Injections of all kinds are as old as computing, as is the unsafe mixing of\ncode and data. <a href=\"#user-content-fnref-inject\" data-footnote-backref=\"\" aria-label=\"Back to reference 3\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-safety\">\n<p>I think of safety in general, not just security. They are deeply\nintertwined, yet the actual, meaningful property we should strive to deliver\nupon has to be safety; security is a critical and related, but not\nidentical, attribute. <a href=\"#user-content-fnref-safety\" data-footnote-backref=\"\" aria-label=\"Back to reference 4\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-macro\">\n<p>At a macro scale this gets complex, and we see plenty of imbalances in who\npays for these externalities; it's why we have regulations for some things,\nfrequently the result of socialized negative outcomes we want to avoid\nrepeating. <a href=\"#user-content-fnref-macro\" data-footnote-backref=\"\" aria-label=\"Back to reference 5\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n</ol>\n</section>",
      "summary": "Brief notes on tools we choose to use, their impacts and risks, and who bears the cost.",
      "date_published": "2026-02-12T18:13:53.452Z",
      "date_modified": "2026-02-12T18:13:53.452Z",
      "tags": [
        "ai",
        "llm",
        "security"
      ]
    },
    {
      "id": "https://hackd.net/posts/mimestream-private-push/",
      "url": "https://hackd.net/posts/mimestream-private-push/",
      "external_url": "https://mimestream.com/trust/private-push",
      "title": "Mimestream Private Push",
      "content_html": "<p>A nice bit of technical detail on how to achieve a privacy-preserving\nimplementation for email push. I hope many people would care about such details,\nthough realistically I'm unsure of it. It's one of those things that can be done\nquickly, easily, and poorly; or with a little more effort and attention, quite\nwell.</p>",
      "date_published": "2026-02-07T02:19:10.320Z",
      "date_modified": "2026-02-07T02:19:10.320Z",
      "tags": [
        "privacy"
      ]
    },
    {
      "id": "https://hackd.net/posts/capabilities-for-agent-delegation/",
      "url": "https://hackd.net/posts/capabilities-for-agent-delegation/",
      "external_url": "https://niyikiza.com/posts/capability-delegation/",
      "title": "Capabilities for agent delegation",
      "content_html": "<p>This is one of the more interesting ideas I've come across in terms of securing\nagentic workloads.</p>",
      "date_published": "2026-01-09T00:55:52.102Z",
      "date_modified": "2026-01-09T00:55:52.102Z",
      "tags": [
        "ai",
        "llm",
        "security"
      ]
    },
    {
      "id": "https://hackd.net/posts/atuin-and-tailscale-and-containers-and-1password/",
      "url": "https://hackd.net/posts/atuin-and-tailscale-and-containers-and-1password/",
      "title": "Atuin and Tailscale and containers and 1Password",
      "content_html": "<p>I'm not sure how it took me this long to discover<sup><a href=\"#user-content-fn-1\" id=\"user-content-user-content-fnref-1\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">1</a></sup> <a href=\"https://atuin.sh\">atuin</a> but I've been\nmissing out for sure. Atuin is essentially (much) \"better shell history\" in a\nbunch of ways, as it:</p>\n<ol>\n<li>uses a database to enable more kinds of queries on the history</li>\n<li>has per-directory (or workspace) contextual filters, to better limit what is\nshown</li>\n<li>syncs your shell history across devices, via an end-to-end encrypted sync\nserver</li>\n</ol>\n<p>That last feature was one of the things I was most excited about, although 2 is\nreally great as well in practice. While there is a public sync server available,\nI thought I should run and host my own, given that's an option and I'm that sort\nof person.</p>\n<p>So, we're setting up a self-hosted Atuin sync server reachable via Tailscale,\nrunning in containers (Podman, but Docker would work), with 1Password providing\nsecrets management, running on macOS. The completed configuration is available\n<a href=\"https://github.com/axtl/atuin-self-server\">on GitHub</a>, if you want to skip ahead. The documentation from Atuin is\npretty thorough, but I did have to figure out a bit more glue, hence this post.</p>\n<h2>Containers (Podman, Docker etc.)</h2>\n<p>I use Podman as most container workloads don't need root. In this particular\nsetup, we'll deploy 3 containers via <code>podman-compose</code>:</p>\n<ol>\n<li>Atuin server</li>\n<li>the Postgres database for the Atuin server</li>\n<li>Tailscale routing so we can reach the Atuin server from our tailnet</li>\n</ol>\n<p>Setting up the Postgres and Atuin containers is pretty well-covered in the\n<a href=\"https://docs.atuin.sh/self-hosting/docker/#docker-compose\">official self-hosting docs</a>, though I did have to figure out a\nsmall thing and ended up submitting a <a href=\"https://github.com/atuinsh/docs/pull/87\">PR</a> to fix the upstream docs.</p>\n<h3>Atuin server</h3>\n<p>Annotated container config:</p>\n<pre><code class=\"language-yaml\">atuin:\n  image: ghcr.io/atuinsh/atuin:latest\n  # start after the DB and Tailscale\n  depends_on:\n    - db\n    - tailscale\n  restart: always\n  command: server start\n  # Make config persistent\n  volumes:\n    - './config:/config'\n  # Map the ports\n  ports:\n    - 8888:8888\n  environment:\n    # Listen an all interfaces.\n    ATUIN_HOST: '0.0.0.0'\n    # same port as above\n    ATUIN_PORT: 8888\n    # I don't keep registration open generally, but you probably need to set\n    # this to \"true\" once to set up a first account.\n    ATUIN_OPEN_REGISTRATION: 'false'\n    # These env vars will be injected later. Make sure that `db` matches your\n    # database container's name!\n    ATUIN_DB_URI: postgres://${ATUIN_DB_USERNAME}:${ATUIN_DB_PASSWORD}@db/${ATUIN_DB_NAME}\n    RUST_LOG: info,atuin_server=debug\n  network_mode: service:tailscale\n</code></pre>\n<p>The main trouble I had was the DB container name not matching the <code>ATUIN_DB_URI</code>\nstring. Now that the upstream docs are fixed, it's unlikely you'd run into this\nissue.</p>\n<p>Given that we're using Tailscale to access the server, I don't worry about\nsetting up TLS.</p>\n<h3>Postgres database</h3>\n<p>Annotated container config:</p>\n<pre><code class=\"language-yaml\">db:\n  image: postgres:17\n  depends_on:\n    - tailscale\n  restart: unless-stopped\n  volumes:\n    # Don't remove permanent storage for index database files!\n    - './database:/var/lib/postgresql/data/'\n    # If the WAL ever gets corrupt (e.g., due to abrupt container stops), fix it with:\n    # docker run -it -v ./database:/var/lib/postgresql/data/ postgres:17 /bin/bash\n    # su postgres &#x26;&#x26; cd /var/lib/postgresql/data &#x26;&#x26; pg_resetwal\n  environment:\n    POSTGRES_USER: ${ATUIN_DB_USERNAME}\n    POSTGRES_PASSWORD: ${ATUIN_DB_PASSWORD}\n    POSTGRES_DB: ${ATUIN_DB_NAME}\n</code></pre>\n<h3>Tailscale</h3>\n<p>We want our Tailscale container to be able to route to and from our Atuin\nserver, while being able to independently authenticate to the network so that we\ndon't lose connectivity periodically. Tailscale docs are pretty good on <a href=\"https://tailscale.com/blog/docker-tailscale-guide\">how to\nset up containers</a>, including OAuth configuration.</p>\n<p>Annotated container config:</p>\n<pre><code class=\"language-yaml\">tailscale:\n  image: ghcr.io/tailscale/tailscale:latest\n  restart: always\n  # TS_AUTHKEY from the OAuth config\n  environment:\n    TS_AUTHKEY: ${TS_AUTHKEY}\n    TS_HOSTNAME: atuin\n    TS_STATE_DIR: /var/lib/tailscale\n    TS_EXTRA_ARGS: --advertise-tags=tag:container\n  volumes:\n    - './tailscale:/var/lib/tailscale/'\n  devices:\n    - /dev/net/tun:/dev/net/tun\n  cap_add:\n    - net_admin\n</code></pre>\n<h2>1Password</h2>\n<p>I've known for a while that 1Password has a CLI integration, allowing secrets\naccess from the command line, in scripts etc. This seemed like a good spot to\ntry that. My approach was to reference the secrets as environment variables (via\n<a href=\"http://mise.jdx.dev\">mise</a>), have their values be paths into a 1Password vault, and start my\ncluster by having <code>op</code> inject the values at start-up time.</p>\n<p>So, for example, to set the database password for Atuin, in <code>mise.toml</code> I have:</p>\n<pre><code class=\"language-toml\">[env]\nATUIN_DB_PASSWORD = \"op://Personal/atuin/password\"\n</code></pre>\n<h2>All together now (hopefully)</h2>\n<p>With the finished <code>compose.yaml</code> for the cluster, and having set up the\nrespective secrets in 1Password, all that's left to do is:</p>\n<pre><code class=\"language-shell\">op run -- podman compose up -d\n</code></pre>\n<p>This should prompt for authentication with 1Password, and then bring up all the\ncontainers and have everything ready to go! As mentioned, you may need to allow\nfor registration once, to set up one account that you'll use. Or keep it on if\nyou're doing this for a team, friend group, polycule, etc. Given the server has\nto be reached over Tailscale, it won't matter too much if you leave this on.</p>\n<p>If you have issues, check the logs for clues. I've had the occasional issue with\nthe DB WAL.</p>\n<pre><code class=\"language-shell\">podman compose logs -n -t -f\n</code></pre>\n<p>You can also configure 1Password integration (and <code>mise</code> populating the env) is\nworking as expected by printing the environment unmasked:</p>\n<pre><code class=\"language-shell\">op run --no-masking -- printenv # might want to pipe into grep ATUIN\n</code></pre>\n<h2>Atuin client</h2>\n<p>Get Atuin (the client) from the <a href=\"https://atuin.sh\">official place</a> or use <code>homebrew</code> /\nanother package manager of your choice. You'll need a little configuration to\ntell your client which is the right server to use.</p>\n<p>Here's my (annotated) configuration as an example, but only the <code>sync_address</code>\nis really needed. For the rest, find what works for you, and make sure to look\nat the <a href=\"https://docs.atuin.sh/self-hosting/docker/#docker-compose\">Atuin docs</a>:</p>\n<pre><code class=\"language-ini\"># Tailnet address and correct port; remember this is HTTP.\nsync_address = \"http://atuin.&#x3C;your-tailnet>.ts.net:8888\"\n# Workspaces use per-git-repository history, rather than directory-only\nworkspaces = true\n# Prefer to see the workspace history by default.\nfilter_mode_shell_up_key_binding = \"workspace\" # or global, host, directory, etc\n# Run commands directly; tab to edit.\nenter_accept = true\n# Use Ctrl-0 .. Ctrl-9 instead of Alt-0 .. Alt-9 UI shortcuts; better on macOS.\nctrl_n_shortcuts = true\n# Invert window?\ninvert = false\n# Window style\nstyle = \"auto\"\n</code></pre>\n<p>At this point you should be able to run <code>atuin status</code> and see you're green and\nconnecting to the server you've just configured!</p>\n<section data-footnotes class=\"footnotes\"><h2 class=\"sr-only\" id=\"user-content-footnote-label\">Footnotes</h2>\n<ol>\n<li id=\"user-content-user-content-fn-1\">\n<p>It was probably\n<a href=\"https://hachyderm.io/@b0rk@jvns.ca/114083615859693297\">this comic</a> by Julia\nEvans that finally made me look. <a href=\"#user-content-fnref-1\" data-footnote-backref=\"\" aria-label=\"Back to reference 1\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n</ol>\n</section>",
      "summary": "Self-hosting an Atuin sync server on macOS (feat. Tailscale, Podman containers, 1Password)",
      "date_published": "2025-06-19T17:47:24.000Z",
      "date_modified": "2025-06-19T17:47:24.000Z",
      "tags": [
        "howto"
      ]
    },
    {
      "id": "https://hackd.net/posts/macos-reflective-code-loading-analysis/",
      "url": "https://hackd.net/posts/macos-reflective-code-loading-analysis/",
      "title": "macOS reflective code loading analysis",
      "content_html": "<h2>Background</h2>\n<p><a href=\"https://attack.mitre.org/techniques/T1620/\">Reflective code loading</a> is an interesting attack technique, useful when\nthere's a desire to conceal or protect the code that's being executed on a\nsystem. This is primarily accomplished by avoiding the creation of any files on\ndisk (e.g., downloading the binary that's then going to run) or other execution\nartifacts that would become indicators of behavior. Instead, code can be\ndownloaded and executed directly in the memory of an otherwise benign process.\nWhile not completely interchangeable, sometimes we see this technique also\nreferred to as \"fileless\", \"artifact-free\", and/or \"in-memory (only)\" as those\nare desirable characteristics.</p>\n<p>Probably the best known implementation of this technique on macOS relies on\nfunctionality exposed by <code>dyld</code>, and first documented in <a href=\"https://www.wiley.com/en-us/The+Mac+Hacker%27s+Handbook-p-9780470395363\">\"The Mac Hacker's\nHandbook\"</a> by Dino Dai Zovi and Charlie Miller in 2009, though it's been\nalso discussed/popularized by <a href=\"https://www.blackhat.com/docs/us-15/materials/us-15-Wardle-Writing-Bad-A-Malware-For-OS-X.pdf\">Patrick Wardle at BlackHat 2015</a>, and by\n<a href=\"https://vimeo.com/215195101\">Stephanie Archibald at INFILTRATE '17</a>. The high-level approach is:</p>\n<ol>\n<li>receive code over a socket</li>\n<li>if it's a binary, change a field in the Mach-O header from <code>MH_EXEC</code> to\n<code>MH_BUNDLE</code> - needed for the subsequent steps as the necessary APIs expect a\nbundle<sup><a href=\"#user-content-fn-bundle\" id=\"user-content-user-content-fnref-bundle\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">1</a></sup></li>\n<li>use <code>NSCreateObjectFileImageFromMemory</code> to, well, create an object file from\nthe memory region that contains the binary</li>\n<li>use <code>NSLinkModule</code> to link in the necessary shared libraries</li>\n<li>call functions or pass execution to the loaded code (after figuring out the\nentry point, in the case it was actually a binary)</li>\n</ol>\n<p>This technique has been used by malware authors rather recently, notably the\n<a href=\"https://objective-see.com/blog/blog_0x51.html\">Lazarus Group in the 2019 version of Apple Jeus</a>.</p>\n<p>There's just one thing: this technique hasn't been truly fileless for… some\ntime.</p>\n<h2>How it (really) works today</h2>\n<p><a href=\"https://twitter.com/bluec0re\">@bluec0re</a> initially got me looking into this in\ndepth, after noticing during a debug run that artifacts <strong>are</strong>, in fact,\ncreated when these specific functions are called. Given the public perception\naround this in-memory loading technique, this was a bit of a surprise, so I\nspent some time doing a bit more research and putting it together for this\nanalysis.</p>\n<p>I used <a href=\"https://twitter.com/its_a_feature_\">@its_a_feature</a>'s simple\n<a href=\"https://github.com/its-a-feature/macos_execute_from_memory\">macos_execute_from_memory</a> PoC for testing (there's also Archibald's\n<a href=\"https://github.com/CylanceVulnResearch/osx_runbin\">PoC</a>, for historical reference). For convenience/ease of\niteration, these PoCs do actually load a target binary from disk, rather than\nover the network, but the execution is meant to be in-memory only.</p>\n<p>To figure out what's going on, a good starting place is to run the code while\nprinting anything that has to do with <code>dyld</code>, given that's where the two\nfunctions are. There are some helpful environment variables to set before\nrunning the code:</p>\n<pre><code class=\"language-fish\">clang -g -o main main.c\nDYLD_PRINT_APIS=1 DYLD_PRINT_LIBRARIES=1 ./main\n</code></pre>\n<pre><code class=\"language-c++\">// some output ellided\ndyld[80184]: NSCreateObjectFileImageFromMemory(0x105010000, 0x00008258)\ndyld[80184]: NSCreateObjectFileImageFromMemory() copy 0x105010000 to 0x10501c000\ndyld[80184]: NSLinkModule(0x6000029d4270, module)\ndyld[80184]: dlopen(\"/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-VTd6S38q\", 0x80000080)\ndyld[80184]: &#x3C;DDBBB7CE-78F7-3E78-AD09-2C79ED030E2A> /private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-VTd6S38q\ndyld[80184]:       dlopen(NSCreateObjectFileImageFromMemory-VTd6S38q) => 0x20a24d4a0\ndyld[80184]: NSLinkModule(0x6000029d4270, module) => 0x20a24d4a0\ndyld[80184]: NSLookupSymbolInModule(0x20a24d4a0, _execute)\ndyld[80184]: NSLookupSymbolInModule(0x20a24d4a0, _execute) => 0x105037f84\nold timey mode: Executed!\ndyld[80184]: NSUnLinkModule(0x20a24d4a0)\ndyld[80184]: dlclose(0x20a24d4a0)\ndyld[80184]: NSDestroyObjectFileImage(0x6000029d4270)\n</code></pre>\n<p>Looks like <code>NSLinkModule</code> uses <code>dlopen</code> to load the\n<code>NSCreateObjectFileImageFromMemory-VTd6S38q</code> file from a temporary location. To\nconfirm and get a bit more detail, I monitored the execution of <code>./main</code> using\nPatrick Wardle's <a href=\"https://objective-see.com/products/utilities.html\">File Monitor</a> to see what files are being accessed. To\nnarrow the output, I'm piping this through <code>jq</code> to filter out for events related\nto the PoC, and extracting only the type and the file destination for each\nevent:</p>\n<pre><code class=\"language-fish\">sudo /Applications/FileMonitor.app/Contents/MacOS/FileMonitor | \\\njq  '.\n   | select(.file.process.path | contains(\"macos_execute_from_memory/main\"))\n   | {event: .event, dest: .file.destination}' | \\\ntee filemon-filtered.json # also saving for futher analysis\n# start ./main in another terminal, Ctrl-C the FileMonitor process when done\n</code></pre>\n<pre><code class=\"language-json\">{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_OPEN\",\n  \"dest\": \"…/macos_execute_from_memory/test.bundle\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_CLOSE\",\n  \"dest\": \"…/macos_execute_from_memory/test.bundle\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_CREATE\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_WRITE\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_CLOSE\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_OPEN\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_CLOSE\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n// open-close twice more\n{\n  \"event\": \"ES_EVENT_TYPE_NOTIFY_UNLINK\",\n  \"dest\": \"/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-7zEgh32K\"\n}\n</code></pre>\n<p>During the execution of <code>./main</code> a file is created, accessed a few times, then\ndeleted. The name changes slightly with every run (the <code>7zEgh32K</code> above,\ncompared to <code>VTd6S38q</code> previously) and based on the path it's fairly obvious\nthis is a temporary file. I wanted to understand more about when it's created,\nand if its presence means this particular technique is no longer as interesting\nas we once thought: it can be replaced with a much less brittle combo of\n<code>dlopen</code> and <code>dlsym</code>, but it's also not artifact-free.</p>\n<p>So next I ran <code>./main</code> in <code>lldb</code> with a breakpoint on\n<code>NSCreateObjectFileImageFromMemory</code>, stepping through instructions to figure out\nwhat's going on:</p>\n<pre><code class=\"language-fish\">(lldb) b NSCreateObjectFileImageFromMemory\nBreakpoint 1: where = libdyld.dylib`NSCreateObjectFileImageFromMemory, address = 0x0000000180314fa0\n(lldb) r\nProcess 94604 launched: 'macos_execute_from_memory/main' (arm64)\nProcess 94604 stopped\n* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1\n    frame #0: 0x000000018e3b8fa0 libdyld.dylib`NSCreateObjectFileImageFromMemory\nlibdyld.dylib`NSCreateObjectFileImageFromMemory:\n->  0x18e3b8fa0 &#x3C;+0>:  mov    x3, x2\n    0x18e3b8fa4 &#x3C;+4>:  mov    x2, x1\n    0x18e3b8fa8 &#x3C;+8>:  mov    x1, x0\n    0x18e3b8fac &#x3C;+12>: adrp   x8, 366208\nTarget 0: (main) stopped.\n</code></pre>\n<p>After for a while we eventually get to\n<code>dyld4::APIs::NSCreateObjectFileImageFromMemory(void const*, unsigned long, __NSObjectFileImage**)</code>\nwhich superficially looks like the function called in <code>main.c</code> but, as we'll see\na bit later on, is not.</p>\n<p>Continuing on we reach first\n<code>libdyld.dylib\\</code>NSCreateObjectFileImageFromMemory<code>and then</code>dyld`dyld4::APIs::NSLinkModule(__NSObjectFileImage*,\nchar const*, unsigned\nint)<code>in a similar fashion. Here we'll see a call to</code>mkstemp` that's a give-away\nwe're likely creating a temporary file:</p>\n<pre><code class=\"language-fish\">(lldb)\nProcess 97220 stopped\n* thread #1, queue = 'com.apple.main-thread', stop reason = instruction step into\n    frame #0: 0x000000018e3bdea8 libdyld.dylib`dyld4::LibSystemHelpers::getenv(char const*) const\nlibdyld.dylib`dyld4::LibSystemHelpers::getenv:\n->  0x18e3bdea8 &#x3C;+0>: mov    x0, x1\n    0x18e3bdeac &#x3C;+4>: b      0x18e3c329c               ; symbol stub for: getenv\n\nlibdyld.dylib`dyld4::LibSystemHelpers::mkstemp:\n    0x18e3bdeb0 &#x3C;+0>: mov    x0, x1\n    0x18e3bdeb4 &#x3C;+4>: b      0x18e3c336c               ; symbol stub for: mkstemp\nTarget 0: (main) stopped.\n(lldb)\nProcess 97220 stopped\n* thread #1, queue = 'com.apple.main-thread', stop reason = instruction step into\n    frame #0: 0x000000018e3bdeac libdyld.dylib`dyld4::LibSystemHelpers::getenv(char const*) const + 4\nlibdyld.dylib`dyld4::LibSystemHelpers::getenv:\n->  0x18e3bdeac &#x3C;+4>: b      0x18e3c329c               ; symbol stub for: getenv\n\nlibdyld.dylib`dyld4::LibSystemHelpers::mkstemp:\n    0x18e3bdeb0 &#x3C;+0>: mov    x0, x1\n    0x18e3bdeb4 &#x3C;+4>: b      0x18e3c336c               ; symbol stub for: mkstemp\n\nlibdyld.dylib`dyld4::LibSystemHelpers::getTLVGetAddrFunc:\n    0x18e3bdeb8 &#x3C;+0>: adrp   x16, -5\nTarget 0: (main) stopped.\n</code></pre>\n<p>Right after the return from <code>mkstemp</code>, I checked the temporary file location to\nsee a file created there:</p>\n<pre><code class=\"language-fish\">fd NSCreateObjectFileImageFromMemory /private/var/folders/\n/private/var/folders/q8/28dylf2973q_22bqzy2_lplr0000gn/T/NSCreateObjectFileImageFromMemory-wySJJ7WJ\n</code></pre>\n<p>The SHA-256 checksum of this temp file matched the SHA-256 checksum of the file\nthat was loaded in-memory by the PoC, so it's pretty clear at this point that\n<code>NSLinkModule</code> writes the file out, then uses <code>dlopen</code> to load it back to do the\nrest. <code>dlopen</code> is part of a stable API, unlike the long-deprecated\n<code>NSCreateObjectFileImageFromMemory</code> and <code>NSLinkModule</code>.</p>\n<h2>Tracing the source code</h2>\n<p>Apple publishes <a href=\"https://github.com/apple-oss-distributions/dyld\">the source code to dyld</a>, though with some delay\nafter the corresponding OS release that introduces a new version. For example,\nthe latest available source is <a href=\"https://github.com/apple-oss-distributions/dyld/tree/rel/dyld-852\">852</a>, which corresponds to macOS 11, but\nthe current macOS 12.2 version is <code>dyld-941.5</code>.</p>\n<pre><code class=\"language-fish\">strings /usr/lib/dyld | rg \"dyld-\\b\\d\\d\\d\\b\" # also /System/Library/dyld/dyld_shared_cache_arm64e\n@(#)PROGRAM:dyld  PROJECT:dyld-941.5\n</code></pre>\n<p>An interesting fact is that the function we saw earlier as\n<code>dyld4::APIs::NSCreateObjectFileImageFromMemory</code> is from the <code>dyld4</code> namespace,\nwhich is not present in any of the source code releases (there's only <code>dyld</code> and\n<code>dyld3</code> respectively.) During debugging I saw a few functions called that are\npart of the <code>dyld3</code> namespace, so my guess is that all of these will coexist, at\nleast for a while. For the purposes of this analysis, I'll focus on the\n<a href=\"https://github.com/apple-oss-distributions/dyld/tree/rel/dyld-852\">852</a> version of <code>dyld</code>, but I'll keep an eye out for whenever 941 drops,\nto check out <code>dyld4</code> there.</p>\n<p>There are a few implementations of <code>NSCreateObjectFileImageFromMemory</code> in the\nsource for <a href=\"https://github.com/apple-oss-distributions/dyld/tree/rel/dyld-852\">852</a>, in:</p>\n<ul>\n<li><a href=\"https://github.com/apple-oss-distributions/dyld/blob/rel/dyld-852/src/dyldAPIs.cpp#L904\"><code>src/dyldAPIs.cpp</code></a>\nadded with the <code>dyld-43</code> release, about 17 years ago.</li>\n<li><a href=\"https://github.com/apple-oss-distributions/dyld/blob/rel/dyld-852/src/dyldAPIsInLibSystem.cpp#L778\"><code>src/dyldAPIsInLibSystem.cpp</code></a>\nadded by <code>dyld-519.2.1</code> in 2017</li>\n<li><a href=\"https://github.com/apple-oss-distributions/dyld/blob/rel/dyld-852/dyld3/APIs_macOS.cpp#L94\"><code>dyld3/APIs_macOS.cpp</code></a>\nwhere the relevant portion was also added with <code>dyld-519.2.1</code> in 2017.</li>\n</ul>\n<p>The same is true for <code>NSLinkModule</code>, just a few lines below in those same files.</p>\n<p>It looks like <code>gUseDyld3</code> controls whether the version that's being used is the\none in the <code>dyld3</code> namespace, versus the \"original\" from the <code>dyld-43</code> release.\nThere are a few places this is assigned to throughout the code base, though only\none spot where it's set to <code>true</code>, in\n<a href=\"https://github.com/apple-oss-distributions/dyld/blob/rel/dyld-852/dyld3/libdyldEntryVector.cpp#L61-L82\"><code>libdyldEntryVector.cpp</code></a>.\nFollowing the path further through the build config files in Xcode and the\nsource itself, I believe the entry vector is a dependency to building\n<code>libdyld.dylib</code>. So that's how we get to the code path that uses the new\n(<code>dyld3</code>) implementation of <code>NSCreateObjectFileImageFromMemory</code> and\n<code>NSLinkModule</code>, the latter of which creates a temp file and loads it via\n<code>dlopen</code>.</p>\n<h2>How long has this been the case?</h2>\n<p>I admit that, while researching, I went through a few different theories as to\nwhen this behavior changed. Given I hadn't seen anyone discuss it, seemed like a\nrecent change. But when I began tracing through the <code>dyld</code> source, I figured\nmaybe this changed around 2017 when the initial <code>dyld3</code> code was added, and\nsomehow nobody noticed. I even went to search VirusTotal for files named\naccording to this pattern, and found quite a few, dating back to 2018.</p>\n<p><img src=\"./rcl-vti-tmp.png\" alt=\"VirusTotal search results for name:NSCreateObjectFileImageFromMemory*\"></p>\n<p>Somehow though, nobody noticing just… didn't seem right?</p>\n<h3>\"Carbon\" dating <code>dyld</code></h3>\n<p>While version 941.5 isn't out, I was curious to check out how execution of this\ncode path looks now, so I opened my local copy of <code>dyld</code> in Ghidra. Searching\nfor <code>NSLinkModule</code> returned a single location, in the <code>dyld4</code> namespace as we\nexpected. The decompiled<sup><a href=\"#user-content-fn-demangler\" id=\"user-content-user-content-fnref-demangler\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">2</a></sup> code does bear resemblance to the source in\nversion\n<a href=\"https://github.com/apple-oss-distributions/dyld/blob/rel/dyld-852/dyld3/APIs_macOS.cpp#L147-L214\">852</a>:</p>\n<pre><code class=\"language-c++\">void dyld4::APIs::NSLinkModule(__NSObjectFileImage*, char const*, unsigned int)\n\n{\n  int iVar1;\n  char *pcVar2;\n  size_t sVar3;\n  undefined8 uVar4;\n  long *plVar5;\n  char acStack1096 [1024];\n  long local_48;\n\n  local_48 = ___stack_chk_guard;\n  if (*(char *)(param_1[1] + 0x90) != '\\0') {\n    dyld4::RuntimeState::log(param_1,\"NSLinkModule(%p, %s)\\n\");\n  }\n  if (param_2[1] == (char *)0x0) {\n    uVar4 = 0;\n  }\n  else {\n    *param_2 = (char *)0x0;\n    pcVar2 = (char *)(**(code **)(*(long *)param_1[0xd] + 0x80))((long *)param_1[0xd],\"TMPDIR\");\n    if ((pcVar2 == (char *)0x0) || (sVar3 = _strlen(pcVar2), sVar3 &#x3C; 3)) {\n      _strlcpy(acStack1096,\"/tmp/\",0x400);\n    }\n    else {\n      _strlcpy(acStack1096,pcVar2,0x400);\n      sVar3 = _strlen(pcVar2);\n      if (pcVar2[sVar3 - 1] != '/') {\n        _strlcat(acStack1096,\"/\",0x400);\n      }\n    }\n    _strlcat(acStack1096,\"NSCreateObjectFileImageFromMemory-XXXXXXXX\",0x400);\n    iVar1 = (**(code **)(*(long *)param_1[0xd] + 0x88))((long *)param_1[0xd],acStack1096);\n    if (iVar1 != -1) {\n      pcVar2 = (char *)_pwrite(iVar1,param_2[1],(size_t)param_2[2],0);\n      if (pcVar2 == param_2[2]) {\n        plVar5 = (long *)param_1[0xd];\n        sVar3 = _strlen(acStack1096);\n        pcVar2 = (char *)(**(code **)(*plVar5 + 8))(plVar5,sVar3 + 1);\n        *param_2 = pcVar2;\n        _strcpy(pcVar2,acStack1096);\n      }\n      _close(iVar1);\n    }\n    uVar4 = 0x80000080;\n  }\n  if (*param_2 != (char *)0x0) {\n    pcVar2 = (char *)(**(code **)(*param_1 + 0x70))(param_1,*param_2,uVar4);\n    param_2[4] = pcVar2;\n    if (pcVar2 != (char *)0x0) {\n      pcVar2 = (char *)dyld4::Loader::loadAddress(dyld4::RuntimeState&#x26; pcVar2 >> 1,param_1);\n      param_2[3] = pcVar2;\n      if (param_2[1] != (char *)0x0) {\n        _unlink(*param_2);\n      }\n      if (*(char *)(param_1[1] + 0x90) != '\\0') {\n        dyld4::RuntimeState::log(param_1,\"NSLinkModule(%p, %s) => %p\\n\");\n      }\n      pcVar2 = param_2[4];\n      goto LAB_00029c40;\n    }\n    if (*(char *)(param_1[1] + 0x90) != '\\0') {\n      (**(code **)(*param_1 + 0x80))(param_1);\n      dyld4::RuntimeState::log(param_1,\"NSLinkModule(%p, %s) => NULL (%s)\\n\");\n    }\n  }\n  pcVar2 = (char *)0x0;\nLAB_00029c40:\n  if (___stack_chk_guard != local_48) {\n                    /* WARNING: Subroutine does not return */\n    ___stack_chk_fail(pcVar2);\n  }\n  return;\n}\n</code></pre>\n<p>Line 32 confirms the template for the temporary name is the same as what we saw\nduring execution.</p>\n<p>Line 51 would be around where we expect a <code>dlopen</code> call to happen, although in\nthis case there's an indirection to resolve an address for a function from a\njump table. Unfortunately Ghidra isn't able to resolve that table, so we're left\nspeculating if that'll end up being <code>dlopen</code> or not. But I'm pretty sure that's\nwhere we end up eventually, given what we've observed during live debugging.</p>\n<p>I wanted to compare this version with some of the older ones, but lacking at the\nmoment any non-Monterey installs, I had to resort to grabbing copies of\n<code>/usr/lib/dyld</code> from VMs and, eventually, download them from VirusTotal. I was\nquite surprised to see that every one of them, as far back as 551, and including\n852, do in fact use a non-<code>dlopen</code> version of the code, basically the true\nfileless implementation. I can't rule out that maybe all of these versions are\noff VMs, where maybe something is different with <code>dyld</code>… I think it's rather\ncurious that I didn't find any that use the <code>dyld3</code> implementation.</p>\n<p>The thing that throws me off the most—if this behavior really was only enabled\nin Monterey—is that so many files matching the <code>mkstemp</code> pattern of\n<code>/private/var/tmp/NSCreateObjectFileImageFromMemory-XXXXXX</code> show up in\nVirusTotal since 2018. I'm not convinced either way just yet.</p>\n<h2>Future work</h2>\n<p>The predictable file name and location means XDR can easily grab these files\nwhenever they're created, as it would be exactly the code to run that's written\nout (so any obfuscation would happen before, most likely). There are some benign\nuses of these APIs, certainly, but there are bound to be some cool findings too.\nI haven't spent any time looking through such files on VirusTotal to see if\nanything good is there, but it's certainly tempting.</p>\n<p>Once the code for the latest version of <code>dyld</code> is out, I hope to better\nunderstand the specifics of how the <code>dyld4</code> namespace is enabled for use. Of\ncourse, I also want to see what direction Apple is taking for these APIs, given\nthe new namespace.</p>\n<p>On the offensive tooling side, I'm curious if it's possible to develop a pure\nin-memory (true fileless) way to execute code on macOS. Mostly that seems to\nrequire writing a Mach-O loader, and I bet significant parts of the logic can be\nborrowed from both the deprecated code, and the <code>dlopen</code> implementation.</p>\n<p>If you happen to come across any interesting implementations of true in-memory\nloaders for macOS, have figured out some of the missing parts I didn't get to in\nthis analysis, or found any issues/mistakes etc. please let me know via\n<a href=\"https://twitter.com/roguesys\">@roguesys</a>.</p>\n<p>Thanks for reading!</p>\n<section data-footnotes class=\"footnotes\"><h2 class=\"sr-only\" id=\"user-content-footnote-label\">Footnotes</h2>\n<ol>\n<li id=\"user-content-user-content-fn-bundle\">\n<p>If we're to go by Apple's old documentation, the motivation for this API is\nto allow loading of plug-ins in applications that might want them. Bundles\n(in the Mach-O sense) fit this purpose rather well, though since it's\ntrivial to turn a binary into a bundle as shown <a href=\"#user-content-fnref-bundle\" data-footnote-backref=\"\" aria-label=\"Back to reference 1\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-demangler\">\n<p>The GNU demangler that's part of Ghidra threw a bunch of errors and did not\ndemangle any names, so I ran the decompiled code through <code>c++filt</code> manually\nand adjusted as necessary. Still, I may have made some errors re:\nparameters, which was fine for my purposes but may not be for yours. <a href=\"#user-content-fnref-demangler\" data-footnote-backref=\"\" aria-label=\"Back to reference 2\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n</ol>\n</section>",
      "summary": "Fileless, artifact-free, in-memory code execution… not quite.",
      "date_published": "2022-02-08T02:45:24.000Z",
      "date_modified": "2022-02-08T02:45:24.000Z",
      "tags": [
        "mac"
      ]
    },
    {
      "id": "https://hackd.net/posts/reorder-arch-headers-universal-macho/",
      "url": "https://hackd.net/posts/reorder-arch-headers-universal-macho/",
      "title": "Reordering Architecture Headers in a Universal Mach-O Binary",
      "content_html": "<p>When I started trialling VS Code and ended up\n<a href=\"/posts/vscode-licensing-ext/\">distracted reverse-engineering extensions</a>, my\ngoal was to document a simple mechanism to re-order the way architectures are\npresented to the loader in universal Mach-O binaries. This is that post.</p>\n<p>Why would this be necessary? For me, it was the need to test a custom Mach-O\nloader, to make sure it can handle certain special cases. This exercise ended up\nbeing a straightforward, practical introduction to the Mach-O file format, which\nin some ways was more valuable, any why I decided to share the process.</p>\n<h2>Intro to universal Mach-O binaries</h2>\n<p>This post focuses on universal (a.k.a. \"fat\") Mach-O binaries, unless otherwise\nnoted. Specifically, those containing object files for <code>x86_64</code> (Intel) and\n<code>arm64</code><sup><a href=\"#user-content-fn-arm64\" id=\"user-content-user-content-fnref-arm64\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">1</a></sup> (Apple Silicon) architectures. There are other potential\narchitectures inside such binaries, since Apple also used the format when moving\nfrom PowerPC to Intel. As we'll see, the specific architectures don't really\nmatter when it comes to reordering, but since I did not test with older\nbinaries, I won't claim this works as-is for those combinations.</p>\n<p>The header describing universal binaries is defined in\n<a href=\"https://github.com/apple/darwin-xnu/blob/main/EXTERNAL_HEADERS/mach-o/fat.h\"><code>mach-o/fat.h</code></a>,\nand the essential information is rather short:</p>\n<pre><code class=\"language-c++\">#define FAT_MAGIC       0xcafebabe\n#define FAT_CIGAM       0xbebafeca      /* NXSwapLong(FAT_MAGIC) */\n\nstruct fat_header {\n        uint32_t        magic;          /* FAT_MAGIC */\n        uint32_t        nfat_arch;      /* number of structs that follow */\n};\n\nstruct fat_arch {\n        cpu_type_t      cputype;        /* cpu specifier (int) */\n        cpu_subtype_t   cpusubtype;     /* machine specifier (int) */\n        uint32_t        offset;         /* file offset to this object file */\n        uint32_t        size;           /* size of this object file */\n        uint32_t        align;          /* alignment as a power of 2 */\n};\n</code></pre>\n<p>Each universal binary starts with a header that is always stored in\n<a href=\"https://github.com/apple/darwin-xnu/blob/main/EXTERNAL_HEADERS/mach-o/fat.h#L36-L37\">big-endian</a>\nformat on disk, and which contains the magic number (<code>0xcafebabe</code>) and the total\nnumber of architectures the binary contains. Headers for each architecture then\nfollow, specifying the CPU, its subtype, an absolute offset into the binary for\nthe object file corresponding to the architecture, that object file's size, and\nan alignment.</p>\n<p>To list architectures in a binary on macOS, either the <code>file</code> utility, or\n<code>lipo -archs</code> can be used; the output is different, but both parse and display\nthe headers, in the order they're present in the binary.</p>\n<h2>Reordering architecture headers</h2>\n<p>The order as displayed is in fact entirely determined by how these headers are\nlaid out inside the binary. Think of the universal binary as a container for its\nmultiple architectures. Each one of those is a (non-universal) binary in its own\nright. In fact, <code>lipo</code> can be used to \"thin\" a binary to only a specified\narchitecture, which essentially parses these headers, identifies the one\ncorresponding to the architecture that was requested, skips to the offset\nindicated in the header, and dumps the next <code>size</code> bytes out.</p>\n<pre><code class=\"language-text\">┌────────────────────────────┐\n│ ┌──────────┐               │\n│┌┤fat_header├──────────────┐│\n││└──────────┘              ││\n││┌────────────────────────┐││\n│││magic                   │││\n││├────────────────────────┤││\n│││num archs               │││\n││└────────────────────────┘││\n│└──────────────────────────┘│\n│ ┌────────────┐             │\n│┌┤arch_headers├────────────┐│\n││└────────────┘            ││\n││┌──────────────────────┬─┐││\n│││ cputype              │0│││\n│││ cpusubtye            └─┤││\n│││ offset                 │││\n│││ size                   │││\n│││ align                  │││\n││├──────────────────────┬─┤││\n│││ cputype              │1│││\n│││ cpusubtye            └─┤││\n│││ offset                 │││\n│││ size                   │││\n│││ align                  │││\n││└──────┬───────┬─────────┘││\n││       │   …   │          ││\n││       └───────┘          ││\n│└──────────────────────────┘│\n└────────────────────────────┘\n</code></pre>\n<p>This means that, in order to reorder the way architectures in the binary are\nprocessed, the simplest approach is to reorder the headers to get the desired\nsequence. It is possible (but insufficient) to shift the actual architecture\ncode in the binary as well, but there's no benefit, except a (dubious)\nperformance claim that if the offset is earlier, there's less seeking necessary\nto get to the relevant code. That's out of scope for this post, but certainly\nsomething one can try.</p>\n<p><code>lipo</code> has a way to carve out architectures from a binary, and also supports\ncreating a universal binary from specific architecture binaries, but the\nordering is fixed, based on a sort by alignment, to save space<sup><a href=\"#user-content-fn-lipo-create\" id=\"user-content-user-content-fnref-lipo-create\" data-footnote-ref aria-describedby=\"user-content-footnote-label\">2</a></sup>, so\nit's unsuitable for reordering. The best option is a small tool that can shift\nthe arch headers as needed. Fortunately, we don't need to <em>understand</em> the arch\nheaders (not that it's hard) for this task, it's sufficient to figure out how\nmany there are, read 20 bytes for each, and save to a new file with a different\nordering.</p>\n<p>The <a href=\"/samples/macho-header-swap.py\">Python code</a> is fairly straightforward,\nthough it doesn't allow for anything fancy like specifying what order the\narchitectures should be output in etc. Simply shifts \"left\", so for example for\nthe <code>file</code> binary, which has the following 3 architectures on my macOS install,\nthe first shift would go from <code>0 1 2</code> to <code>1 2 0</code> etc. as indicated below:</p>\n<pre><code class=\"language-text\">arch     | x86_64 arm64 arm64e -> arm64 arm64e x86_64 -> arm64e x86_64 arm64\nposition |    0     1      2   ->   1      2      0   ->    2      0     1\n</code></pre>\n<p>Obviously (and reassuringly) shifting all the way to the original yields the\nsame SHA256 hash for the binary.</p>\n<p>To begin, read in the <code>fat_header</code> to confirm this has multiple architectures,\nthen read how many architectures we're dealing with:</p>\n<pre><code class=\"language-python\">offset = 0\nmagic = struct.unpack(\">I\", inbin.read(4))[0]\noffset += 4\ninbin.seek(offset)\n\nif magic != MAGIC:\n    print(\"not a universal binary\")\n    return 2\n\n# next value tells us how many archs the binary contains\nnarchs = struct.unpack(\">I\", inbin.read(4))[0]\noffset += 4\ninbin.seek(offset)\n\nif narchs &#x3C; 2:\n    print(\"not enough archs: %d\", narchs)\n    return 3\n</code></pre>\n<p>Once we know how many architectures are in the binary, we can simply read their\nheaders, 20 bytes at the time, but we do not need to parse them. This would be\ndifferent if we also had to reorder the object files inside the binary, since\noffsets would need to be adjusted, but that's not the case here.</p>\n<pre><code class=\"language-python\">headers = []\nfor _ in range(narchs):\n    headers.append(inbin.read(20))\n    offset += 20  # promise\n    inbin.seek(offset)\n</code></pre>\n<p>Then write to the output binary, maintaining the <code>fat_header</code> and using a simple\nshift in the loop to write the arch headers themselves. Finally, copy the\nremainder of the file:</p>\n<pre><code class=\"language-python\"># write back magic\noutbin.write(struct.pack(\">I\", magic))\n# write out how many narchs\noutbin.write(struct.pack(\">I\", narchs))\n# put headers in, shifting \"left\"\nfor idx in range(1, narchs + 1):\n    outbin.write(headers[idx % narchs])\n# inneficiently copy the remaining bytes\noutbin.write(inbin.read())\n</code></pre>\n<h2>Final notes</h2>\n<p>There is not much to this once there's an understanding of how the headers in\nthe binary are laid out. This is somewhat documented in the open source code\nApple publishes, though resources like Jonathan Levin's books make for much\neasier references to learn from.</p>\n<p>As for follow-up work (besides improved error checking) a better way to\nunderstand the Mach-O format is to also enable object file reordering in the\nbinary, and/or have a way to specify the sequence for header reordering.</p>\n<section data-footnotes class=\"footnotes\"><h2 class=\"sr-only\" id=\"user-content-footnote-label\">Footnotes</h2>\n<ol>\n<li id=\"user-content-user-content-fn-arm64\">\n<p>Frequently this will in fact be <code>arm64e</code>, but in most cases discussed here,\nit is not necessary to make the distinction. <a href=\"#user-content-fnref-arm64\" data-footnote-backref=\"\" aria-label=\"Back to reference 1\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n<li id=\"user-content-user-content-fn-lipo-create\">\n<p>Search for <code>* create_fat</code> to find the function in\n<a href=\"https://opensource.apple.com/source/cctools/cctools-973.0.1/misc/lipo.c.auto.html\">lipo.c</a> <a href=\"#user-content-fnref-lipo-create\" data-footnote-backref=\"\" aria-label=\"Back to reference 2\" class=\"data-footnote-backref\">↩</a></p>\n</li>\n</ol>\n</section>",
      "summary": "A simple way to re-order platform architectures in universal Mach-O binaries.",
      "date_published": "2021-07-08T16:24:49.000Z",
      "date_modified": "2021-07-08T16:24:49.000Z",
      "tags": [
        "mac"
      ]
    }
  ]
}