Aviation Software Testing Culture: Why ‘It Works’ Is Never Enough

There is exactly one word between ‘the software works’ and ‘the software has been proven to work’ — but the operational and legal distance between those two statements is enormous. The development team delivered on time. The operations team adopted the system without complaint. Management signed off. Everything looks fine. Then, six months later, an EASA auditor asks for test documentation on the current release, and the answer is ‘we tested it, but we did not keep records.’ At that point, months of engineering effort are worth nothing in the audit room. Aviation testing culture exists precisely because this gap between doing and proving is where accidents and compliance failures are born.Turkey’s civil aviation sector is expanding rapidly in 2011. Turkish Airlines is deepening its international network, Pegasus is building out domestic routes, and ground handling companies like Havaş are scaling their operational capacity. Every layer of that growth rests on a growing software foundation: flight planning systems, maintenance tracking applications, ground services coordination platforms, reservation infrastructure. A testing gap in any of those systems — not from negligence but from the absence of a structured habit — can produce serious consequences before both SHGM (Turkey’s Directorate General of Civil Aviation) and EASA. The core argument here is direct: in aviation software, testing is not primarily about finding bugs. It is about constructing a chain of accountability. If that chain is not on paper, functional excellence tells the auditor nothing.The most commonly missed distinction is the difference between testing a working system and documenting that a system has been tested. Consider a representative scenario: a ground services company based in Ankara with 264 employees deploys a new baggage tracking system. The vendor’s team spends months on the product, runs test scenarios internally, closes several critical defects. At handover, the system genuinely works. But the test scenarios were not written in a standard format, there is no record linking which scenario ran against which version, and defect reports are scattered across an email thread rather than a tracking system. When the SHGM audit arrives, there is no trail to present. The system works — but it has not been verified. Those are not the same thing, and aviation regulators treat them very differently.The operational risk of untraceable testing grows in a far more insidious way than functional risk. When a production incident occurs — a mislabelled baggage route, a maintenance interval calculation error, a flight time change that does not propagate to the notification layer in time — the first investigative question is always: ‘Was this scenario tested?’ Without a test record, the answer defaults to no. ‘We did it but did not write it down’ is not an acceptable answer under aviation standards. The DO-178B framework, which governs software verification in aeronautical systems, treats the documentation of a verification activity as equivalent to the activity itself: an undocumented test is a test that did not happen. Full adoption of DO-178B is not yet universal across Turkish operators in 2011, but every company working with EASA-certified components feels this standard’s pressure indirectly. The test discipline you build today is the one that keeps you from doing expensive retroactive rework when formal compliance requirements tighten further.Scenario breadth is the second major gap in aviation testing culture, and it tends to be the more systematic blind spot. Teams test the expected flow: data arrives correctly, the system processes it correctly, output appears in the right format. That ‘happy path’ test is necessary but nowhere close to sufficient. In an aviation environment, real failure hides in unexpected input combinations, at system boundary values, and under cascading error conditions. Take a maintenance tracking software project at an Istanbul-based operator: the system handles single flight records perfectly, but when two maintenance entries are opened simultaneously for the same tail number, the system silently overwrites one record without producing an error message. That kind of edge case scenario — concurrent state mutation with no conflict detection — only surfaces in production if it was never added to the test scenario library before go-live. The breadth of that library is a direct measure of the testing process’s maturity.The difference between embedding quality into a process from the start and inspecting for it at the end is the first discipline to collapse under schedule pressure. When project timelines compress, the testing phase gets treated as a compressible buffer. But the relationship between defect discovery cost and project phase is not linear. A defect caught during requirements is dramatically cheaper to fix than the same defect caught after go-live; in aviation software, the post-production multiplier is steeper because a production failure is not just a software problem — it can mean an operational halt or a formal regulator notification. At a mid-size Turkish airport operator last year, compressing testing into the final three days before a ground services software update led to a critical reporting defect surfacing 48 hours after go-live, requiring two days of manual override operations. The total recovery cost exceeded what a ten-day extended test period would have cost by a significant margin. That is not a hypothetical; it is the pattern that repeats across projects where testing is treated as an end-of-pipeline gate rather than an embedded discipline.As the manager responsible for a software project in aviation operations, three questions should be on your desk before the next release. First: are your test scenarios written in a standard format — scenario ID, version reference, input conditions, expected output, actual result, sign-off — or do you have only an informal record that ‘we tested it’? Second: what is the ratio of happy-path scenarios to boundary and error-condition scenarios in your test library? If the answer skews heavily toward the former, your scenario coverage is underestimating where aviation systems actually break. Third: are your test records linked to your version control, so that for any given release you can show precisely which scenarios were executed, which defects were closed, and who approved the sign-off? The answers to those three questions reveal your testing culture’s actual maturity in minutes. In aviation, ‘it works’ is the starting point. ‘It has been proven to work, and here is the evidence’ is the only acceptable finishing line. That baggage tracking system in Ankara is still running fine — but when the next audit comes around, will the team be ready to prove it?

This article was originally published in Turkish by Gökhan MERCANOĞLU on January 10, 2011. The English edition has been reviewed and edited by the author.


For master data management, the critical question is not which system to use. The real question is which problem will be solved, which data can be trusted, and which action will be accelerated. Without these answers, solutions look modern but only digitize old habits.


Gökhan Mercanoğlu
ERP ve Kurumsal Yazılım