Slides / 19 of 19
Talk slides
From audio and video data to evidence: AI analytics, biometrics, and spoofing and deepfake detection
Mikhail Inkin · STC, Speech Technology Center
Day 2 — Friday, 12 September 2025: information security day · Scheduled 17:20–17:50 · In the recording 08:50:23–09:26:43
These slides are an English version of the Russian deck shown in the recording: the text the authors set in the presentation has been translated and redrawn in place, while everything captured from a screen — program windows, logs, evidence — is left exactly as it was, in Russian. The fonts are the closest available ones, not the originals. The Russian deck is on the RU version of this page. Under each slide its text is set out for word search and machine reading.

Page 108:51:41 Slide text
| ГРУППА
(C) ИРТ
КОМПАНИЙ
MOSCOW FORENSICS DAY '25 12.09.25
From audio and video data to evidence
AI analytics, biometrics, spoofing and deepfake detection
Speaker: Mikhail Inkin Head of project group
16+

Page 208:52:09 Slide text
About us
(О) ЦРТ
• ГРУППА КОМПАНИЙ
The STC group of companies develops products and solutions based on intelligent speech technologies, generative artificial intelligence, and face and voice biometrics
Saint Petersburg Moscow
One of the largest players in the AI market in Russia
30+
5000+
projects worldwide, including nationwide projects
years in the AI market
Winner of international scientific competitions in speech recognition and biometrics
CHiME
NIST
ASVSPOOF
CHALLENGE
750+
employees on staff
70+
R&D teams
20
PhDs and doctors of science
Talent pool
ViTMO
Корпоративная лаборатория Технологии человеко-машинного взаимодействия
www.speechpro.ru
2

Page 308:52:35 Slide text
LLM solutions from Sber
(С) ЦРТ КОМПАН-
КОМПАНИЙ
Business process transformation
8794
Knowledge base search
Margin income calculation
Predictive equipment diagnostics
Procurement assistant
GIGAIDE
8582
VERSE
www.speechpro.ru

Page 408:52:53 Slide text
Problems
Ever-growing volumes of unstructured data
(С) ЦРТ
ГРУППА КОМПАНИЙ
Low efficiency of data search and processing
High cost of scaling data processing systems
Low speed of analysis and decision-making
Loss of important and valuable information
www.speechpro.ru

Page 508:53:31 Slide text
Types of biometric systems
(C) ЦРТ
ГРУППА КОМПАНИЙ
System types
Automatic
Automated
Expert (manual)
Language-independent
Language-dependent
Text-independent
Text-dependent
GMM-UBM Xvector ResNet Wav2Vec (HuBERT)
Microphone type: near-field mid-field far-field
www.speechpro.ru
5

Page 608:54:17 Slide text
Human speech synthesis
(С) ЦРТ КРУПАН
КОМПАНИЙ
Every person's voice is unique!
More than 70 body parts are involved in producing the voice
Not what we say matters, but who speaks
More than 30 years of use in forensic examination
Hard Palate
Soft Palate (Velum)
Pharyngeal Cavity
Larynx
Esophagus
Trachea
Nasal Cavity
- Nostril
• Lip Tongue Teeth Oral Cavity
Jaw
• Lung
Diaphragm
www.speechpro.ru

Page 708:54:35 Slide text
Speaker model
(С) ЦРТ КОМПАН-
кОМПАнИЙ
Feature vector, embedding
V1, V1,2 ... Vi,m
V2,1 V2,2... V2n
V3,1 V3,2 ... V3,n
Vn.л Vn,2 ••. Vn.л
www.speechpro ru

Page 808:54:56 Slide text
Identifying characteristics of the voice
(С) ЦРТ
ІГРУППА КОМПАНИЙ
Fundamental frequency (F0) (pitch, fundamental tone) — the rate of vocal fold vibration, is determined by the vocal tract characteristics and lies in the ranges (60-180 Hz M, 160-350 Hz F, and 200-650 Hz for children).
Formants — frequency maxima arising from the resonance of the human vocal tract. Usually three formants (F1, F2, F3) are enough both to distinguish sounds and to recognise the speaker. Every formant is a function of all parts of the vocal tract, though the degree of their influence is not the same in each case.
8

Page 908:55:52 Slide text
Challenges of the task
(О) ЦРТ
| ГРУППА КОМПАНИЙ
Causes of data variability
Reading, spontaneous speech, monologue, dialogue, group conversation
Inter-speaker
variability
Intra-speaker
variability
Noise, SNR
Microphone
Vocabulary
Accent, dialect
Speaker-related
Technological
Communicative
Stress
Emotions
Lombard effect
Language/Culture
[1] - Speaker recognition by machines and humans [Hansen J.H.L., Hasan T., 2015]
9

Page 1008:56:30 Slide text
Approaches to speaker comparison
(С) ЦРТ КОМПАНИЙ
Auditory
Acoustic-phonetic
Spectrographic: comparison of formants, F0, F0 harmonics
Automatic
www.speechpro.ru

Page 1108:57:00 Slide text
Solving a practical case

Page 1208:57:32 Slide text
Product line and the stages of the forensic process
(С) ЦРТ КРУПА
КОМПАНИЙ
Data collection
Data analysis
Evidence
Decision
Multipasport Nezabudka
Gnom
• AVIS
IKAR Lab 3
• Nestor.Al
www.speechpro.ru
13

Page 1308:58:21 Slide text
AVIS
(С) ЦРТ
ГРУППА КОМПАНИЙ
Products for public safety
Biometric search
Diarization
Full-text search
Metadata search
Gender, language
Semantic search
Auto-translation
Cluster analysis
Speech to text
Statistical analysis
Spoofing/deepfake detector
Link analysis
GOG
Scattered data
Practical result
Video identification systems
ON
Public safety institutions
Biometric data
Person profile enrichment
14

Page 1408:58:51 Slide text
What happens to the data in AVIS
(С) ЦРТ КОМПАН
кОМПАНИЙ
Рабочий стол
www.speechpro.ru

Page 1508:58:59 Slide text
What happens to the data in AVIS
(С) ЦРТ
кОМПАНИЙ
Аналиа
www.speechpro.ru

Page 1608:59:06 Slide text
What happens to the data in AVIS
(С) ЦРТ
ГРУППА кОмПАНИЙ
Анализ
www.speechpro.ru

Page 1708:59:13 Slide text
What happens to the data in AVIS
• ГРУППА
(С) ЦРТ
КОМПАНИЙ
Аналиа
www.speechpro.ru
15

Page 1808:59:27 Slide text
What happens to the data in AVIS
() ЦРТ
КОМПАНИЙ

Page 1908:59:45 Slide text
What happens to the data in AVIS
(С) ЦРТ
• ГРУППА
КОМПАНИЙ
www.speechpro.ru

Page 2008:51:16 Slide text
Biometric link graph
(С) ЦРТ КРУПА
КОМПАНИЙ
www.speechpro.ru

Page 2109:00:24 Slide text
LLM in solving forensic tasks
(С) ЦРТ КОМПЛА
КОМПАНИЙ
Text generation from instructions
Question answering
Text classification
Personalisation of answers
Creating coherent and grammatically correct text on a given topic
Providing answers to questions based on information known to the LLM and on context
Determining categories or labels for a given text (for example, sentiment analysis)
Adapting answers and behaviour to the specific user
Information extraction
Conversation support
Code generation and debugging
Text summarisation
Picking out key facts and data from text
Holding dialogues with users, understanding and contextualising previous turns
Writing code and fixing errors in it in various programming languages from a task description
Condensed rendering of long texts, picking out the main ideas
Text translation
Translating text from one language into another
Data interpretation
Explaining data analysis results, data visualisation
www.speechpro.ru
19

Page 2209:00:43 Slide text
Bulk information processing
КОМПАН
AVIS
Поиск в архиве
panta
0 0 00000
30а 01 33 34
www.speechpro.ru

Page 2309:00:55 Slide text
Bulk information processing
() ЦРТ
| ГРУППА КОМПАНИЙ
AVIS
Поиск в архиве
MeTE
www.speechpro.ru
20

Page 2409:01:04 Slide text
Bulk information processing
(0) ЦРТ
Т ГРУППА КОМПАНИЙ
AVIS
Анализ
• 23000
00 10.54
DU1AT1
3018 10
Dances sou
www.speechpro.ru
20

Page 2509:01:11 Slide text
Bulk information processing
МПАН
AVIS
Анализ
2 santos
Q. Беседы зази
• XS090
301411
Dance 30 crocume
301211
www.speechpro.ru
20

Page 2609:01:25 Slide text
Bulk information processing
(0) ЦРТ
• ГРУППА
КОМПАНИ
AVIS
Анализ
Q Беселы рази
ame
Iligence Repor
www speechpro.ru
20

Page 2709:01:32 Slide text
Bulk information processing
(0) ЦРТ
| ГРУППА КОМПАНИЙ
AVIS
Анализ
9 беседы рази
0 230
20

Page 2809:02:08 Slide text
Auto-annotation of individual recordings
(С)) ЦРТ КРУПАН
КОМПАНИЙ
Детали
Результат работы Al-агента
Call summary_ Eng
О дати
Отчет по Анализу Звонка
1. Название Звонка: Обсуж 2. Дата Звомка: Не 3. Ключевы
зднике" (birthday party) 06
•(cake coming through this ероприятикх (petting the big lesques").
Be' ("propert)
ительность мероприятия
ениет
Тейст
Эрок / Время
Дополнительная Информан
Отсутствует точная дата звони
альшого дня" ('big day)
шего анализа
www.speechpro.ru
22

Page 2909:02:38 Slide text
ГРУППА
(C) ИРТ
КОМПАНИЙ
Exporting recordings to SIS
Анализ
1 Рост уведомления
Data KZ + KW
Аудиозаписи
Статистикл
дунио записи
+ Дата события
2023-01-24
2135
Выгрузить аудио
mato -tos
104101
Mena
2023-01-24
000251

Page 3009:03:02 Slide text
Preliminary analysis
(О) ЦРТ
ГРУППА КОМПАНИЙ
www.speechpro.ru
24

Page 3109:03:20 Slide text
Feature extraction and identification
(С) ЦРТ | КОМПАНИЯ
Automatic comparison algorithms
Speech and noise detection
Editing and processing of recordings
Identification wizard
Expert comparison methods
Signal characteristics calculation
Tho bas ho bo ba
Audio visualisation and analysis
EdiTracker module
HO DH
Ti bà bà bà ba bie
Authenticity diagnostics
Expert report
41 4
“Super Loupe” module
25

Page 3209:04:10 Slide text
(С) ЦРТ КОМПАН
КОМПАНИЙ
Example 2

Page 3309:06:03 Slide text
Spoofing detector
(С) ЦРТ
ГРУППА
КОМПАНИЙ
The detection module* estimates the probability of spoofing (synthesis/conversion, replay, overall score) in the recording.
- A.Ri
wa - ElevenLatu, 2125-08-23710,34,44, Egor 12
Постовеная состиагающия
An overall score and separate scores by attack type are calculated.
• Mapod
13.09 секуто
Three spoofogram graphs are produced, showing the sliding score values against the signal time. If the synthesis probability score is 50% or above, the extra function for identifying the synthesis vendor can be enabled.
* Supplied under a separate licence
27

Page 3409:07:09 Slide text
Spoofing check or authenticity check
(С) ЦРТ
І ГРУППА КОМПАНИЙ
Проверка аутентичности
A spoofing check — identifying and detecting forged media files with algorithms that analyse inconsistencies and with deepfake detection technologies.
Проверка спуфинт»
Врализ по.
An authenticity check — confirming the originality and integrity of digital content:
metadata analysis; checking digital signatures and watermarks; comparison with trusted sources; deepfake detection.
Анализ
Deepfake detection aims to find forged media files, whereas an authenticity check confirms the originality and integrity of digital content.
www.speechpro.ru

Page 3509:07:51 Slide text
History of spoofing
(С) ИРТ ГРУППА
• КОМПАНИЙ
"SH" whistle.
Nostrils
Speech sounds
come out here..)
Resonator
of leather'
'SH" lever
Reed cut-off
"S"* lever
Bellows,
Auxiliary bellows
"S" whistle
Leather
Nostril
- Reed
Compressed
mina
air chamber
Mark J. Jones, Rachael-Anne Knight (Eds.) The Bloomsbury Companion to Phonetics. 2013
www.speechpro ru
29

Page 3609:08:54 Slide text
Some examples
(О) ЦРТ КРУПА-
КОМПАНИЙ
1920
1979
Fidelity Voice Chess Challenger (1979)
the first talking chess computer
https://www.ismenio.com/ chess fidelity vcc.html
2000
A simple speech generator
https://ieeexplore.ieee.org /document/861820
2023
2024
Fake Joe Biden robocall urges New Hampshire voters not to vote in Tuesday's Democratic
00
Simple AI voice cloning
Advanced voice cloning systems with emotions
Google search: voice clone
https://www.cnn.com/2024/01/ 22/politics/fake-joe-biden- robocall/index.html
31

Page 3709:09:57 Slide text
SIS capabilities
(C)) ЦРТ
ГРУППА КОМПАНИЙ
Expert method for detecting spoofing
Spoofing assessment
The automatic detector produces an initial score for the recording
The expert records spoofing features in an interactive artefact table*
The result — a report saved into the project
* Supplied under a separate licence
www.speechpro.r

Page 3809:10:30 Slide text
SIS capabilities
ГРУППА
()) ЦРТ
І КОМПАНИЙ
EdiTracker module
EdiTracker makes it possible to find signs of a breach of authenticity using the following methods:
Examining the technical characteristics of the recording device
Indicators of Dighal Processing Spr
6. 5.512. 8. 11.025. 16 KH2
the spectrum contains a fal off typicaly caured by and alas flering at lection rehabilly: medum
Detecting traces of digital signal pre-processing
Detecting stationary harmonics and scanning their phase
Background noise scanning
Auditory analysis
www.speechpro.n

Page 3909:12:13 Slide text
І ГРУППА
(C) ИРТ
КОМПАНИЙ
LOVE
FRIENDSHIP
Example 3
ACCeSS

Page 4009:12:45 Slide text
Deepfake analysis
() ЦРТ КРУПАНИЙ
1 ГРУППА
The video frames of the media file are analysed
A high probability of a deepfake has been determined for the synthetic video Frame-by-frame analysis shows a score for every frame
LOVE
FRIENDSIIP
After the analysis an averaged high probability that a deepfake is present is displayed
www.speechpro.ru

Page 4109:12:52 Slide text
Deepfake analysis
(С) ЦРТ КРУПАНИ
1 ГРУПП
The video frames of the media file are analysed
A high probability of a deepfake has been determined for the synthetic video Frame-by-frame analysis shows a score for every frame
LovE
FRIENDSHIP
Iml
380 конн
After the analysis an averaged high probability that a deepfake is present is displayed
www.speechpro.ru
38

Page 4209:13:34 Slide text
Deepfake analysis
()) ЦРТ
| ГРУППА І КОМПАНИЙ
The video frames of the media file are analysed
A high probability of a deepfake has been determined for the synthetic video Frame-by-frame analysis shows a score for every frame
LOVE
FRIENDSHE
98%
-28.02 граку
After the analysis an averaged high probability that a deepfake is present is displayed
www speechpro.ru
38

Page 4309:14:26 Slide text
Deepfake analysis
(С) ЦРТ
| ГРУППА КОМПАНИЙ
The video frames of the media file are analysed
A low probability of a deepfake has been determined for the original video Frame-by-frame analysis shows a score for every frame
ASCeSS
After the analysis an averaged low probability that a deepfake is present is displayed
www.speechpro.ru
39

Page 4409:14:57 Slide text
SIS capabilities
(О) ЦРТ
І ГРУППА І КОМПАНИЙ
Expert method for detecting deepfakes
The automatic score can be supplemented by expert analysis of deepfake features* in the video frames
The expert uses an interactive artefact table, marking the typical distortions not only on the face but across the whole frame
The result is a report that is saved in the project
* Supplied under a separate licence
40

Page 4509:15:39 Slide text
Nestor.Al
(О) ЦРТ
ГРУППА
| КОМПАНИЙ
destor Al
Admin e
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
0
beting Name
Ites quine
arman. 2106.2025,3415
21.05.2025.34.15 m
588R.audo.3025-02-12,00
sudio.2025-02-18.00-27-44 mar
12-37
wNd0.2023-02-19.00-22-37m0
22-33
SHER. Ando.1021-02-12.00
aс0 2020-02-300-22-28 ман
5869.sudio.2025-02-12,09 22-24
5809.0080.2025-02-18.09-
adi, 2029-02-18.010-21-13 ma
Rec 1
4110)
Строк на стран
www.speechpro
* Supplied under a separate licence
41

Page 4609:16:10 Slide text
Nestor.Al
(С) ЦРТ КОМПЛА
| КОМПАНИ!
lentoc Al
trasnes
Admin 6
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
Ric 1
0000L.1.1 ma
camel
2021-09-22
Pyccon
2025-08-20
2025-08-20
2025-00-54
2025-08-14
Русскии
2023-08-14
Русскин
Русский
2025-08-13
2025-08-13
Prcosi
2025-08-07
2001-04-06
Pyccon
2001-04-08
2025-08-07
Pyccon
2023-08-07
2001-04-00
Arcan
20 - 1-7
Cooa tee
* Supplied under a separate licence
41

Page 4709:16:15 Slide text
Nestor.Al
| ГРУППА
(С) ЦРТ КОМПАНИЙ
destoc Al Framan Tm
Admin e
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
01 L0
2021-09-27 Русский
2025-08-21
2085-00 20
2025-08-20 Русски
2025-09-14
2025-08-54
EFFEEEFD
Русски
2073-08-14
2025-08-14
Русски
61 m
2025-08-14
2025-08-14
Pycoond
2025-08-13
2025-09-13
2025-08-07
2001-04-06
Русски
2025-08-07
Pycoon
on-3a
2001-04-00
2025-08-07
Орокнетіренн 20 - 1-20
* Supplied under a separate licence
41

Page 4809:16:20 Slide text
Nestor.Al
(С) ЦРТ
ГРУППА КОМПАНИЙ
lentor Al
Admis e
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
TUPI OCEA
02.06.207100207
POCOETA MPO
2025-08-21
2025-08-20
2025-09-14
30
2025-08-14
2025-08-14 038
2025-08-13
200
2025-08-07
20
2025-08-07
www.speechpro
* Supplied under a separate licence
41

Page 4909:16:30 Slide text
Nestor.Al
(О) ЦРТ
І ГРУППА
| КОМПАНИЙ
destoc Al Franar
Admic 6
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
OUCO 1
02.0.2071.90.202
Сірок не страния
2025-08-21
2025-08-20
2025-08-14
30
2025-08-14
2025-08-14
2025-09-13
www.speechpro
* Supplied under a separate licence
41

Page 5009:16:35 Slide text
Nestor.Al
(0) ЦРТ |
І ГРУППА
destoc Al Frasa
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
* Supplied under a separate licence
41

Page 5109:16:40 Slide text
Nestor.Al
(О) ЦРТ КРУПА-
КОМПАНИЙ
destoc Al Fase
Admin e
In court proceedings over an audio forgery the Nestor.AI system was used: it automatically turned the speech of the session into text, and the AI assistant prepared a short summary of the content and saved it into a strictly formatted document together with the transcript
Экотирту Speaker зя организовать поездку в офи
Отвентвенне
Срок
оручени
Андерс-
укачено
і зеленые лина!
рганизовить повзаку в орис
мовый позлучный пари
інтерст
садут
одушный шарик в форми
накрос
Спортие поженть
Meeting Minutes
www.speechpro
* Supplied under a separate licence
41

Page 5209:16:54 Slide text
Conclusion
ГРУППА
(С) ЦРТ
КОМПАНІ
The products of the STC group of companies work at different stages of the forensic process and make it possible to use voice biometrics, face biometrics and intelligent speech technologies both to search for the material of interest and to obtain forensic reports
www.speechpro.rul