{
  "author": {
    "id": "person:tony-malott",
    "name": "Tony Malott"
  },
  "canonicalWorkId": "work:complexity-has-to-earn-its-keep",
  "classification": {
    "publicSafe": true,
    "treatmentClass": "flagship",
    "workType": "systems-thesis"
  },
  "discovery": {
    "abstract": "A first-person systems essay arguing for Portable Decision Surfaces, reconstruction packets, least-privilege distribution, and using AI to compile repeated reasoning into deterministic software instead of defaulting to another persistent application stack.",
    "card": {
      "kicker": "Earned complexity",
      "orientation": "Do not ask what backend to build until a requirement proves the simpler architecture insufficient.",
      "visual_motif": "decision-surface-gate"
    },
    "collections": [],
    "format": {
      "depth": "Deep thesis",
      "interaction": "Static editorial reading with technical teaching surfaces",
      "label": "Flagship Work",
      "reading_time": "14 min"
    },
    "kind": "Systems thesis",
    "thesis": "Enterprise decision tools should start with governed data, deterministic portable software, and controlled refresh; persistent backend complexity should be added only when a concrete requirement proves it necessary.",
    "topics": [
      "systems-engineering",
      "enterprise-architecture",
      "deterministic-software",
      "ai-assisted-engineering",
      "sharepoint-governance",
      "inference-debt",
      "portable-decision-surface"
    ]
  },
  "evidence": [
    {
      "id": "evidence:issue-1253",
      "locator": "https://github.com/pinklon/shareplane-platform/issues/1253",
      "role": "Governs public-safe semantics, accepted source custody, and qualified publication.",
      "title": "Issue #1253 publication authority",
      "type": "owner-governing-authority"
    }
  ],
  "lineage": {
    "canonicalUrl": "https://shareplane.malott.ai/artifacts/complexity-has-to-earn-its-keep/",
    "relationship": "new-public-work"
  },
  "provenance": {
    "acceptedEdition": "owner-accepted-v1-self-contained",
    "disclosureBoundary": "No employer, internal product, teammate, hostname, username, tenant, workstation, or confidential implementation detail is exposed. Portable Decision Surface, portable reconstruction packet, and inference-debt doctrine are presented as original engineering synthesis, not Microsoft-defined patterns.",
    "ownerIssue": 1253,
    "sourceKind": "owner-accepted-self-contained-source",
    "sourcePackageBytes": 32229,
    "sourcePackageSha256": "2d2f4cd8bb08ff6d3f48effbe4228512cab5bad220f883839e0f81eac3b722be"
  },
  "relationships": [
    {
      "explanation": "Both Works argue that stable reasoning should move toward deterministic machinery once ambiguity has been sufficiently reduced.",
      "id": "relationship:complexity-1",
      "predicate": "companion_to",
      "readerGroup": "companions",
      "readerLabel": "Inference at the Ambiguity Frontier",
      "sourceId": "artifact:complexity-has-to-earn-its-keep",
      "standing": "active",
      "targetId": "artifact:inference-at-the-ambiguity-frontier"
    },
    {
      "explanation": "Both Works argue that stronger surrounding control reduces unnecessary frontier reasoning and avoids paying repeatedly for ambiguity the system can already resolve.",
      "id": "relationship:complexity-2",
      "predicate": "companion_to",
      "readerGroup": "companions",
      "readerLabel": "The Most Expensive Model Is Often Compensation for Missing Control",
      "sourceId": "artifact:complexity-has-to-earn-its-keep",
      "standing": "active",
      "targetId": "artifact:the-most-expensive-model-is-often-compensation-for-missing-control"
    }
  ],
  "schemaVersion": "shareplane.work-portable-view.v1",
  "text": "# Complexity Has to Earn Its Keep\n\n## Why enterprise decision software should start simple, use AI to eliminate repeated inference, and add infrastructure only when the requirement proves it belongs\n\nI have built systems that I later wanted to simplify, sometimes aggressively.\n\nThat matters because this is not an argument from the cheap seats. It is easy to look at somebody else's architecture after the boxes have multiplied and conclude that the whole thing has become absurd. It is considerably harder when some of those boxes exist because you put them there yourself.\n\nI have built software because the problem justified it at the time, handed it forward, watched requirements change, watched engineering add capabilities, watched operational obligations accumulate, and then returned later with a much less comfortable question: why are we still carrying all of this?\n\nThere is usually an answer for every component. The database exists because the application needs data. The API exists because the front end needs access to that data. The runtime exists because the API has to execute somewhere. Then come service identities, secrets, deployment pipelines, monitoring, alerting, recovery, patching, dependency management and operational ownership because once the machinery exists, somebody has to keep it alive.\n\nEvery answer can be individually reasonable while the resulting system is collectively unreasonable.\n\nThe problem is not necessarily that somebody made a bad engineering decision. The problem is that organizations are much better at asking why a component should be added than why it should continue to exist.\n\n**Complexity has to earn its keep.**\n\n## We keep starting with the solution\n\nA surprising amount of enterprise software begins with an architectural reflex.\n\nSomeone needs an inventory viewer, readiness console, operational dashboard, evidence browser, architecture assessment, fleet explorer, compliance surface or decision-support tool. The actual requirement is often straightforward: take information the enterprise already possesses, organize it, calculate or correlate some things, make relationships visible, and help a human make a better decision.\n\nThen software culture takes over.\n\nThe discussion shifts quickly to databases, APIs, hosting, authentication, service accounts, secrets, deployment pipelines, observability, recovery and support. Before long, a modest decision tool has inherited the operational obligations of a small commercial application.\n\nSometimes that is exactly right. A transactional system really does need transactionality. Real-time coordination really does need persistent runtime. Privileged integrations, server-side secrets and authoritative mutation need controlled execution somewhere.\n\nThose are requirements.\n\n\"It is an application\" is not a requirement.\n\nThe first architecture question should therefore be more demanding than \"what stack should we use?\"\n\n> **What requirement exists that the simpler architecture cannot satisfy?**\n\nIf we cannot answer that precisely, the complexity has not yet justified itself.\n\n## The enterprise already owns more of the stack than we admit\n\nThis becomes especially interesting inside large enterprises because we repeatedly ignore infrastructure that already exists.\n\nThe organization may already have durable storage, enterprise identity, group-based access control, retention, version history, auditing, search, information classification, lifecycle management and administrative governance. It has already paid for those capabilities, already operates them and already has people responsible for them.\n\nThen a department needs a bounded decision tool and we recreate much of that machinery beside the systems that already contain the authoritative information.\n\nWe copy the data into another store. We create another authorization path. We stand up another runtime. We create another place where information can become stale. We acquire another operational dependency and another thing that needs to survive organizational changes, staff turnover and platform upgrades.\n\nEventually somebody produces a presentation explaining why the application estate has become too complicated.\n\nThere are absolutely cases where the duplication is necessary. The mistake is assuming necessity before proving it.\n\nSharePoint is a useful example because many enterprises already use it for governed document and information storage, but this is not really a SharePoint thesis. The broader principle is simpler: **count the capabilities of the platform you already own before rebuilding them somewhere else.**\n\nIf an existing governed information plane can satisfy persistence, identity, retention, access control and distribution requirements, those are real architectural capabilities. They should be treated as such.\n\nThat changes the design space considerably.\n\n## The Portable Decision Surface\n\nThe pattern I keep coming back to is what I call a **Portable Decision Surface**.\n\nAt its simplest:\n\n**governed source plane → controlled reconstruction packet → AI-assisted engineering → deterministic decision surface → controlled refresh**\n\nThe authoritative information remains where the enterprise already governs it. A controlled process assembles the bounded information needed for a particular decision surface. AI can help interpret messy inputs, reconcile structures, create transformation logic, improve the interface, generate tests and work through ambiguity.\n\nThen the useful result gets turned into deterministic software.\n\nThe resulting surface might be an interactive HTML application, a JavaScript explorer, an evidence browser, an inventory console, a report, a visualization or something more specialized. It may contain a substantial amount of code. It may be sophisticated software.\n\nWhat it does not automatically require is a permanent service behind it.\n\n**Software is not synonymous with service.**\n\nThat distinction is more important than it sounds. We have become accustomed to treating application development and service development as essentially the same activity. They overlap frequently, but they are not identical.\n\nA piece of software does not become more legitimate because a server is continuously running somewhere behind it.\n\nFor many bounded, read-oriented decision tools, the runtime can live almost entirely in the portable artifact while persistence, identity and governance remain in platforms the enterprise already operates.\n\n## The packet matters more than the conversation\n\nAI makes this architecture considerably more practical, but perhaps not for the reason people initially assume.\n\nThe interesting part is not the chatbot. It is portability.\n\nAI sessions are temporary. Context windows end. Execution environments disappear. Models change. Products change. Licensing changes. Vendors change their minds about what a feature should be called this quarter.\n\nThe engineering state does not have to disappear with any of them.\n\nA well-constructed package can carry forward the durable state of the work: source material or governed references, application code, schemas, transformations, assumptions, tests, instructions, provenance and the important decisions already made.\n\nI think of that as a **portable reconstruction packet**.\n\nSometimes the packet is literally a ZIP file. There is nothing profound about the file format. Humanity solved \"put related files together\" long before generative AI showed up and started getting credit for discovering folders.\n\nThe important part is the discipline. The packet should contain enough state that another capable execution environment can understand what exists, reconstruct it, modify it and validate the result without depending on the memory of the previous conversation.\n\nOnce that happens, the AI conversation stops being the unit of continuity.\n\nThe work becomes the unit of continuity.\n\nThat has important consequences. An ephemeral execution environment can be useful without becoming authoritative. One model can start the work and another can continue it. The intelligence can be temporary while the engineering state remains durable.\n\nThat is a much healthier dependency model.\n\n## Use your tokens to burn down inference debt\n\nThis is the part of the pattern I care about most.\n\nA lot of AI architecture seems to assume that progress means putting more inference into the finished system. I often want exactly the opposite.\n\nI want to use expensive intelligence aggressively while the problem is messy and then eliminate as much repeated intelligence as practical once the problem is understood.\n\nUse a model to discover structure in inconsistent inputs. Let it reconcile schemas, identify edge cases, write parsers, generate transformation logic, build tests, improve the interface and explain relationships that were difficult to see before.\n\nThen capture that understanding.\n\nStable decisions should become schemas. Repeatable judgments should become rules. Transformations should become code. Expected behavior should become tests. Important invariants should become deterministic validation.\n\n**Use your tokens to burn down inference debt.**\n\nInference debt is what we create when a system has to keep reasoning about something we already know how to express deterministically.\n\nIf the same judgment will be made fifty thousand times, and we can safely encode it in software after understanding it once, I would rather pay for the reasoning once than pay a model to rediscover the same answer fifty thousand times.\n\nThat is not an argument against AI. It is an argument for using AI where its leverage is highest.\n\nProbabilistic systems are extremely useful against ambiguity, synthesis and incomplete structure. But once ambiguity has been reduced enough that stable behavior can be expressed deterministically, the architecture should be allowed to change.\n\nInference that still earns its keep should remain.\n\nInference that no longer earns its keep should disappear.\n\nThe same standard should apply to every other architectural dependency.\n\n## Refresh is not runtime\n\nA similar category error appears whenever someone says, \"But the data changes.\"\n\nNaturally it changes. Almost all useful operational information changes.\n\nThat does not prove that the user-facing application needs an always-running backend.\n\nThere is a meaningful difference between a **refresh requirement** and a **runtime requirement**.\n\nIf the underlying information changes daily, perhaps the deterministic surface is regenerated daily. If it changes hourly, perhaps the refresh runs hourly. A refresh process can retrieve or receive updated information, validate the structure, apply deterministic transformations, record provenance, build a new package, verify the output and publish the replacement.\n\nIf validation fails, the process can stop rather than publishing a bad state.\n\nThe decision surface itself can remain portable.\n\nFor many internal tools, this is perfectly adequate. Users still receive current information, but the organization does not inherit an always-running application tier whose principal responsibility is staying alive between updates.\n\nThere will be cases where this approach is obviously insufficient. That is fine. Those cases have identified a requirement that earns additional architecture.\n\nThe point is simply that changing data does not automatically imply persistent runtime.\n\n## \"Internal\" is not a security model\n\nThe security conversation also becomes distorted once AI enters the room.\n\nSuppose a dataset contains employee names, usernames, workstation names, device identifiers or operational metadata. That information deserves appropriate protection.\n\nBut none of those information classes appeared when generative AI arrived.\n\nEnterprises have been moving them through spreadsheets, exports, internal websites, reports, ticketing systems, email attachments, shared drives and dashboards for decades. All of those surfaces can be overshared.\n\nAn internal website can be overshared. A spreadsheet can be overshared. A CSV can be overshared. A SharePoint library can be overshared. An AI workflow can be overshared.\n\nThe governance question remains fundamentally the same: who can see the information, why can they see it, what can they do with it, how long should access remain, and what controls actually enforce those decisions?\n\nThat is why \"but it contains employee information\" is not, by itself, a particularly useful architectural objection. The information classification matters, but so does the actual exposure model.\n\nConsider two designs.\n\nOne is an internal dashboard accessible to thousands of employees because that is simply how the site was deployed.\n\nThe other is a deliberately permissioned decision surface available to twenty people who have a defined business reason to see the information.\n\nAssuming the controls are functioning as intended, those are not equivalent risk surfaces. The smaller authorized population represents the narrower exposure.\n\nThis does not mean that a controlled SharePoint folder or any other repository is secure simply because somebody intended it to be. Permissions still have to be correct. Sharing behavior matters. Downloads matter. Retention matters. Auditability matters. Copies remain copies.\n\nThat is ordinary information governance.\n\nAI does not require us to abandon that reasoning. If anything, it gives us another reason to practice it properly instead of relying on the comforting but meaningless label \"internal.\"\n\n**Internal is not the same thing as least privilege.**\n\n## The backend still has a job\n\nNone of this means backends are obsolete. That would replace one reflex with another, which is not much progress.\n\nPersistent services earn their keep constantly.\n\nIf a system needs transactional consistency across concurrent writers, server-side secrets, privileged service identities, authoritative mutations, durable queues, long-running state machines, real-time coordination, centralized policy enforcement, dynamic querying across very large datasets, hard availability requirements or controlled external integrations, persistent infrastructure is probably the correct architecture.\n\nThen build it.\n\nBuild it properly, secure it, monitor it, patch it, recover it and accept the cost of owning it.\n\nThe difference is that the requirement came first.\n\nI do not want no backend.\n\nI want **no unearned backend**.\n\n## Every box creates an obligation\n\nArchitecture diagrams make components look deceptively inexpensive.\n\nThey are rectangles.\n\nReal systems are less accommodating.\n\nEvery rectangle creates a lifecycle. A database needs maintenance and recovery. An API acquires compatibility requirements. A service identity requires credential governance. A runtime needs patching. Dependencies need updating. Queues need failure semantics. Monitoring creates response obligations. Caches eventually demand invalidation strategies because software engineering occasionally appears to have been designed as an endurance test.\n\nNone of those components are inherently bad.\n\nThe point is that **capability and obligation arrive together**.\n\nThis distinction becomes more important as AI reduces the cost of producing software.\n\nA coding model can generate another service remarkably quickly. It can generate the tests, infrastructure configuration, deployment scripts and documentation too.\n\nThe enterprise may still own that service for a decade.\n\nThat means cheap implementation can actually make architectural discipline more important, not less. We are entering an era where organizations can create machinery faster than they can rationally retire it.\n\nThe scarce question is no longer whether the code can be produced.\n\nIt usually can.\n\nThe harder question is whether the thing should exist at all.\n\n## This is not a low-code thesis\n\nPortable and simple should not be confused with primitive.\n\nA Portable Decision Surface can contain serious software. It can support interactive visualization, filtering, search, local analytics, validation, provenance, evidence navigation, accessibility, controlled export and polished user experience.\n\nThe simplification is architectural.\n\nWe are reducing unnecessary runtime dependencies, not reducing engineering quality.\n\nIn some cases I would happily trade more deterministic application code for less infrastructure. Code can be inspected. Code can be tested. Code can be packaged. Code can be versioned. Code can survive the disappearance of the AI system that helped write it.\n\nThat is a useful trade.\n\nThe goal is not less capable software.\n\nThe goal is software whose operational footprint is proportional to the problem being solved.\n\n## The long tail is enormous\n\nDashboards are only the obvious example.\n\nLarge enterprises contain a long tail of internal tools that are primarily projections over information already held somewhere else: audit evidence viewers, migration assessments, fleet inventories, architecture reviews, risk registers, compliance explorers, operational scorecards, capacity tools, readiness packages, historical evidence browsers, portfolio summaries and configuration viewers.\n\nMany are read-heavy. Many serve bounded audiences. Many refresh periodically rather than continuously. Many do not own authoritative transactional state. Their purpose is often to improve human understanding rather than coordinate machines in real time.\n\nNone of those properties proves that a Portable Decision Surface is the right answer.\n\nThey make it worth asking the question.\n\nThat distinction matters because I am not trying to create another architecture religion. Enterprise technology already has a healthy supply of those.\n\nI am proposing a test:\n\n> **What capability does the more complex architecture provide that the simpler one cannot?**\n\nIf the answer is concrete, the complexity earns its keep.\n\nIf the answer is tradition, familiarity or \"this is how we normally build applications,\" the architecture has not made its case yet.\n\n## Architecture should become simpler as understanding improves\n\nThere is also a cultural problem underneath this.\n\nEngineering organizations are good at adding things. Removal feels more dangerous.\n\nA new component solves a visible problem. Removing one creates anxiety about invisible dependencies. Teams change. Original assumptions get lost. Requirements disappear into history. Architecture accumulates through a series of decisions that were often perfectly reasonable when considered individually.\n\nComplexity enters one defensible decision at a time.\n\nEventually nobody can explain the entire system without an hour-long meeting and three diagrams.\n\nMature engineering should push in the other direction too.\n\nAs understanding improves, architecture should have permission to shrink.\n\nWhen AI helps turn fuzzy manual reasoning into deterministic software, the system should become simpler. When an existing governed platform already satisfies identity, retention, access and storage requirements, those capabilities should count. When periodic regeneration satisfies freshness, we should not pretend continuous runtime is necessary.\n\nAnd when a component no longer satisfies a requirement that matters, we should be willing to remove it even if we were the ones who designed it.\n\nEspecially if we were the ones who designed it.\n\nThere should be no architectural prestige attached to preserving machinery that no longer earns its existence.\n\n## Keep it simple until complexity earns it\n\nI am not arguing for simplistic systems.\n\nI am arguing for systems whose complexity is proportional to their requirements.\n\nThat is a much harder standard.\n\nKeep authoritative information in the governed plane when that plane can satisfy the requirement. Carry durable engineering context in reconstruction packets instead of depending on the memory of an AI conversation. Use frontier intelligence aggressively where ambiguity demands it, then capture what it learns and turn stable reasoning into deterministic software.\n\nSeparate refresh requirements from runtime requirements. Control the actual audience instead of treating \"internal\" as a security model. Add persistent services when they provide capabilities the simpler architecture genuinely cannot provide.\n\nAnd every time another box appears on the architecture diagram, ask the same question:\n\n> **What requirement do you satisfy that the simpler architecture does not?**\n\nIf it has a good answer, keep it.\n\nIf it does not, take it out.\n\nAI is going to make it extraordinarily easy to build more software.\n\nThe harder discipline will be knowing what not to build.\n\n**Complexity has to earn its keep.**\n",
  "title": "Complexity Has to Earn Its Keep",
  "workId": "artifact:complexity-has-to-earn-its-keep"
}
