# Automated government systems under the forensic computer expert's microscope: problems and solutions Oleg Bezik · Digital Research Laboratory MOSCOW FORENSICS DAY ’25 · Day 2 — Friday, 12 September 2025: information security day · Scheduled 16:45–17:15 · In the recording 08:13:25–08:50:23 Talk summary · https://2025.moscow-forensics-day.workers.dev/en/summary/19-bezik Transcript: https://2025.moscow-forensics-day.workers.dev/en/transcript/19-bezik · Slides: https://2025.moscow-forensics-day.workers.dev/en/slides/18-bezik-avtomatizirovannye-sistemy · Watch from 08:13:25: https://youtu.be/4V7Wez3L_58?t=29605 --- ## In brief A talk on forensic computer examinations (SKTE) of custom-built, primarily government, automated systems: what questions the court puts, why an examination takes a year or two, why the expert ends up working as a tester. The trigger is a contract of 150 million – several billion rubles and software that does not meet the ToR. The speaker is not selling a product and names the weak points of the practice himself; the second half of the segment is a discussion with a female opponent from the audience. ## Key points - A state automated system (GAS) automates a function of a government body: "Pravosudie" — court proceedings, "Legal Statistics" — prosecutors. - Contracts of 150 million – "several billion rubles"; disputes in the commercial (arbitrazh) courts, sometimes "it even gets to criminal cases"; an SKTE — "99% of the time, even 99.99%". - Appraisers are brought in (the cost of what was done, or of "finishing the system") — the examination becomes multidisciplinary; "I'm not an appraiser" is repeated three times. - The court's questions: compliance with the contract / the ToR / the detailed ToR; what does not work and why; critical defects and whether they are remediable; cost. - The ToR for large systems is detailed, but it can also be ten pages; there can also be a demand beyond the ToR — and the contractor fulfills it for nothing. - A critical defect: "the buttons are blinking", but the report that matters does not get generated — a "pretty toy", people do the work by hand; an irremediable one is a defect whose fixing means a new contract "for another billion rubles". - Pain point No. 1: publicly available methodologies for examining automated systems "simply don't exist"; their own algorithm (objects → completeness of the deliverables → compliance with the ToR) they "haven't packaged". - Every automated system is a "unique palace": "one developer can do it for 100 thousand, another for a million, and a third for a billion". - Pain point No. 2 is volume: boxes of paper, "millions of lines of code", systems for 5–10 thousand people; in the current examination — **1,041 requirements** by hand, "as testers of sorts". - Timeframes — a year, a year and a half, two, "really expensive"; a panel of 3 or 4 people; the record is 9 people in the room: 2 forensic experts and 7 representatives. - Cost — the Moscow DIT and COCOMO methodologies; the scheme "87 percent was done, so that's 87 million" he calls a common one, "but we don't do it that way". - Tips: take stock of the objects, prepare the questions for the expert, hire professional forensic experts; a PDF checklist from the slide. ## Tools, artifacts, technologies - **GAS "Legal Statistics"** — the object of the examination: "from scratch", per the detailed ToR and the **test programme and procedure**; **GAS "Pravosudie"**, **Gosuslugi** — examples of automated systems. - **The Moscow DIT methodology**, **COCOMO** — valuation by quantitative indicators, the results are "more or less the same". - **Their own algorithm for examining automated systems** — not packaged; **software for comparing source code** — their own development since 2025, unnamed; **a checklist (PDF)**. - **Objects**: the ToR, the detailed ToRs, manuals, explanatory notes, contracts; source code on flash drives and discs; the deployed system. - From the discussion: **SAP**, **banking systems**, **the handheld terminal**, **Microsoft Windows**, **aircraft simulators**; **the state information system of Rosreestr** (Barannikov) — acceptance under Federal Law 44-FZ, code "with hashes", load tests. ## Legal and organizational context An SKTE is a way of bringing specialized knowledge into court proceedings; it is often multidisciplinary, with an appraiser ("not an economist as such, but an appraiser"). The venue is the commercial (arbitrazh) courts, sometimes "criminal cases". The court puts the questions; the parties have the right to be present; "the court pays us, not a party". The contract documents: the ToR, the detailed ToR, the test programme and procedure, acceptance. From the audience — Federal Law 44-FZ (acceptance of a state contract with experts involved, as the customer's insurance), the position of the EKC MVD (check the discrepancies rather than test the system), R&D projects, Interpolitex. The disputed point is whether an expert may file a motion to have the questions reformulated. ## Questions from the audience There is no speaker labeling, the remarks from the audience were recognized less well; attribution is by context. - **A former employee of the Forensic Science Centre of the Ministry of Internal Affairs of Russia** (her name is not spoken): at the EKC "never in our lives did we take on a question about testing", the expert's job is the specific discrepancies; Windows or an aircraft simulator cannot be checked against a ToR. → Bezik: the court puts the questions; this is about systems built from scratch — GAS "Legal Statistics", with a detailed ToR and a test programme and procedure. - The argument about reformulating the questions: "You can, why not?" — "I've never come across that. Never"; 5 million for an examination versus "200 thousand, or 500 thousand". - **Sergey Nikolaevich Barannikov**, a forensic expert: the checking has to be done at acceptance under Federal Law 44-FZ, so that the customer does not "go to prison for accepting God knows what"; the case is the state information system of Rosreestr. - **Andrey**: are the cost questions handled in a multidisciplinary format, with economists, or in-house? → Multidisciplinary; what is needed is specifically an appraiser. - **An unnamed participant** (possibly the same Andrey; heavily garbled): what formulas should be used to cost a "super-high-quality" development, and what happens to the system while the dispute is going on? The answer covers the first. - **The same woman**: "you're calculating the percentages all wrong" — the weight of a module cannot be determined; in R&D the winner is whoever has prior developments. The finale: at the tests "90% of the tasks are settled", the dispute is over 10%, "so which of them will pay you those 5 million"? → "I think we're speaking different languages". ## The speaker's position He stays inside the procedural frame: he answers the court's questions as they are put; if the question is about compliance with the ToR, then every requirement gets checked. He dismisses the SAP and Windows argument as being about off-the-shelf products: custom systems with a detailed ToR and a test programme and procedure can be checked in full. He names the limits himself: "I'm not an appraiser", the methodology is not packaged, the examinations are long and expensive. The tone is a lecture with self-irony, defensive in the discussion. ## Quotes - "…99% of the time, even 99.99%, a forensic computer examination gets appointed." - "Well, as they say, without a good spec, the result is anyone's guess." - "…don't take on all the testing. That's a road to nowhere." - "It's always amazed me when people try to evaluate what they don't understand." - "The court pays us, not a party."