blog.arda.tr 100 entries · since 2010
▾
◂ back to the ledger

Digital Necromancy

excerpt

Centralizing a fragmented knowledge hoard by subjugating the APIs of Manning, O'Reilly, and Packt through MCP.

Cover image: Digital Necromancy

I have accumulated enough PDF and EPUB files to train a dozen LLMs or bore a thousand junior developers to death. The problem isn’t the content; it’s the fragmentation. Apress, Packt, Manning, O’Reilly—they all want me to log into their disparate, crumbling walled gardens to access my data.

I do not ask. I take.

Digital Necromancy

This week, I declared war on scattered libraries. The goal: Centralize everything into my local Calibre instance, accessible via a custom Model Context Protocol (MCP) server running from Antigravity.

The Architecture: calibre-mcp

This is not a client-server relationship; it is a surgical intervention. Calibre uses SQLite, and SQLite hates concurrency. You cannot have the application open while the agent operates.

So, the protocol is simple: Calibre must sleep so the Agent may feast.

We established a Model Context Protocol (MCP) server to interface directly with the local metadata.db on the filesystem. Instead of blindly dumping files into a folder and praying for the best, the agent queries the database file directly. It knows what I have. It knows what I lack. It implements a “skip if exists” logic that saves bandwidth and sanity, all without launching the GUI.

{
  "mcpServers": {
    "calibre": {
      "command": "uvx",
      "args": [
        "calibre-mcp",
        "--db-path",
        "C:\\Users\\akara\\Calibre Library\\metadata.db"
      ]
    }
  }
}

The Campaigns

1. Apress (The Warm-up)

Apress was the target dummy. We matched simple text titles to the database to verify the MCP connection. It was boring, compliant, and functioned exactly as a test case should. Coverage verified.

2. Packt (The Bloatware)

Packt is the fast food of technical publishing. Their download links are signed and expire, effectively mocking standard scraping attempts.

  • The Problem: Their library is polluted with thousands of “Video” and “Code” files that I have zero interest in archiving.
  • The Solution: We deployed a “Kill List” (filter_packt.py). We ignored the noise and focused solely on the text.
  • The Breach: Headless browsers (playwright) failed against their Cloudflare gates. The workaround? A direct assault using curl_cffi masquerading as a legitimate browser session via hijacked cookies. The inventory is secured.

3. Manning (The Victory)

Manning tried to be clever with dynamic AJAX dashboards and S3 redirects.

  • The Hack: download_manning.py parses the AJAX endpoint to find the hidden links. When the server redirects to Amazon S3 (which rejects Manning’s cookies), the script intelligently drops the auth headers and requests the resource naked.
  • The Result: ~76 books extracted, downloaded, imported, and retroactively tagged via fix_manning_tags.py. 100% coverage. Clean.

4. O’Reilly (The Behemoth)

No API. Just a massive, scrolling DOM of “Your Products.”

  • The Brute Force: I fed the raw HTML into the machine. download_oreilly.py extracted ~1500 download links from the DOM.
  • Status: Currently processing ~190 files in the background. The script bypasses login checks by injecting a pre-authenticated session from oreilly_cookies.json.

The Loot: Classics Recovered

We filtered thousands of files. Most were discarded—casualties of the framework wars. But amidst the debris, we salvaged the Immutable Core: books that survive because they teach concepts, not syntax.

The Esoteric & Functional Cult

CoverBook
The Joy of ClojureThe Joy of Clojure (Manning)
Not just a language guide, but a philosophy text that rewrites how you think about data.
Real World OCamlReal World OCaml (O’Reilly)
The ultimate status symbol for the functional elitist. Hard, fast, and uncompromising.
Real World HaskellReal World Haskell (O’Reilly)
For when you want to feel superior to everyone else in the room.
Programming ElixirProgramming Elixir (Pragmatic Bookshelf)
Because concurrency shouldn’t hurt this much.

The Deep Magic (Internals & Theory)

CoverBook
Understanding ComputationUnderstanding Computation (O’Reilly)
From simple machines to impossible programs. This is the computer science degree you never finished.
Low-Level ProgrammingLow-Level Programming (Apress)
C, Assembly, and program execution on Intel 64. For when you need to speak directly to the metal.
The Joys of HashingThe Joys of Hashing (Apress)
A niche masterpiece on hash table programming. Pure algorithmic sadism.

The Art of Code

CoverBook
Your Code as a Crime SceneYour Code as a Crime Scene (Pragmatic Bookshelf)
Using forensic psychology and graph theory to analyze technical debt. Essential reading.
Mazes for ProgrammersMazes for Programmers (Pragmatic Bookshelf)
Coding for the sake of beauty and algorithms. A rare gem that sparks joy.
Programming for Musicians and Digital ArtistsProgramming for Musicians and Digital Artists (Manning)
The only file in this entire dump that has a soul.

The Architects

CoverBook
Patterns of Application Development Using AIPatterns of Application Development Using AI (Manning)
The new manifesto. It treats LLMs as architectural components, not magic wands.
Designing GamesDesigning Games (O’Reilly)
By Tynan Sylvester (creator of RimWorld). It discusses the mechanics of emotion, not just Unity scripts.
Metaprogramming Ruby 2Metaprogramming Ruby 2 (Pragmatic Bookshelf)
The spellbook for bending the language to your will.

The library is no longer a scattered mess of vendor-locked web portals. It is a single, searchable monolith. The code serves the user. Not the other way around.

THE END GAME

Some of you may be wondering, “Arda what are you doing ? these are obsolete junk, why bother ?” Well, I now have full access to my library, empowered by a assistant. I can now do things like:

calibre-mcp-1

calibre-mcp-2

I can query via my assistant, have summaries, cross check, inference. It’s like having a research assistant who has read every book in my library.

Introducing tincan: two cans and a string for AI agents ai · dev · go · agents 5 min
Saving the Slimy Lichmummy dev · go · ai · retro 7 min
I put Wikipedia, Stack Overflow and the Amiga Hardware Reference Manual on my NAS, for my agents ai · agents · mcp · homelab · amiga 6 min