Only this pageAll pages
Powered by GitBook
1 of 54

Home

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Welcome to the Home of the vCons

vCons safely and securely carry conversations from the network elements that create them to the applications that analyze them, enabling responsible management of the most personal of data.

vCons are the file format for recording conversations with a "natural person". As a ground truth, vCons enable responsible management of the most sensitive of customer data: the sounds of our voice and the appearance of our face.
The IETF SCITT WG defines a set of interoperable building blocks that will allow implementers to build integrity and accountability into software supply chain systems to help assure trustworthy operation. vCons ❤️ SCITT
The Conserver creates, processes and manages the lifecycle of vCons at scale.
A depiction of an analysis link, a fundamental component of the conserver. Links contain the processing of a vCon. Chains are composed of links, representing workflow in the Conserver. In this example, the Analysis link accepts a vCon as an input, then using an LLM or other AI system, adds a new item to the vCon analysis array to track the interaction.

Quick links

Get Started

We've put together some helpful guides for you to get setup with our product quickly and easily.

Conserver

🌎vCons are...
💬A vCon Primer
🚀Conserver Introduction
💡Concepts
✨More Information
🐰Conserver Quick Start

Deep Dives

Storage

The standard storages supported by the conserver

The conserver supports these standard storages:

  • ChatGPT FIles

  • Elastic Search

  • File

  • Mongo

  • Postgres

  • Redis

  • S3

  • SFTP

Tools

AI Integration

Simplifying Working with Bots

Many programmable communication developers share the same experiences when they begin working on voice bots and transcriptions. They begin by capturing the conversation data with MS Word or Excel files. Often a combination of both to better represent conversations and enable the transcriptions to be processed / cleaned-up.

Later they build a JSON format that includes conversation data, transcriptions, context, etc. Soon they begin writing macros / functions to automate conversation data processing and analysis. With that comes maintaining both the JSON format and the macros / functions, and a realization they do not have the time to build everything themselves. vCon solves this common problem and enables macros / functions to be written to this common format and contributed to the open source project.

vCon Adapters

Service Provider Migration

When companies merge or are acquired, they consolide suppliers to gain economies of scale. However, migrating data from an acquired company's UCaaS / CCaaS onto the acquiring company’s platform can be problematic.

Often the acquired company’s customer data and historical conversations are lost, or an expensive data migration project is undertaken. Businesses discover their customer conversation data is not really their data.

With vCon such a migration becomes an export / import of the historical conversations.

This could also be applied to a personal use case where an individual wants to download all their communications from a social network.

Verbal Agreements - Converted to Writing

As a general rule, the law in the UK considers verbal contracts to be as legally binding as written ones, and therefore they do hold up in court. But this is an exception compared to the rest of the world. Generally, for established business relationships it makes doing business for small projects easy. Often just an email that summarizes the offer, a consideration, and acceptance is enough, the project gets delivered, and everyone is happy.

However, when legal gets wind of such dealings the $5k project that was supposed to be delivered next week doesn’t happen as the ‘standard’ T&Cs require contractor insurance with $X million+ liability coverage that takes more than a week to arrange. Plus all the time and effort in word-smithing far exceeds the project size.

Using a vCon on the call, chat, or email thread to discuss and agree the project, and using an app such as Meeting Minder - Contract Edition, a verbal or text agreement can easily be converted into a written one, with lightweight T&Cs begrudgingly accepted by legal for small projects. Now legal are not unhappy and the business can focus on operations, not document editing, with just a call creating a lightweight contract for review and signing.

Note ASR is not necessarily required in this use case. The conversation could be transcribed by a person, in some medical use cases such manual transcription continues to be required. A contract could have a value to make manual transcription economic or potentially preferred.

Authenticating and Certifying Conversations

A carrier / telco knows both participants of a conversation were in Germany as the call was made over their network by devices physically connected to their network. Certifying the conversation was made in Germany places additional restrictions on how the data can be used to protect both parties. Personal data is constitutionally protected in Germany. This means individuals have the power to decide when and to what extent personal information is published.

To comply with the Dodd-Frank act's call recording regulations, companies must keep all communication records made through the telephone, voicemail, email, and others, and these records must be uniformly time stamped. A carrier could provide this as a service, recording all conversations made by a business’s mobile phones, and sharing with the business as a vCon. The conversation could additionally be processed by the carrier or CPaaS/UCaas/CCaaS to confirm no PII was shared in this conversation, or certify no keywords specified by the business were found. A business would only want to pay once for that service and attach the certification to the vCon.

Phone numbers can be spoofed. But within a carrier’s network they have knowledge of the device’s identity, its location, its phone number, its SIM (Subscriber Identity Module) card identity, the owner’s account, how long they have had that number or device, etc. There is no other agency that can confirm the identities of the parties on a call with such confidence. A carrier could certify the identities on a vCon.

HELPS and HACKS

MCP Server

Overview

Please note most of these use cases are voice-centric, we see voice conversations as an initial opportunity. However, vCon works for conversations across any communications media, e.g. email, video, SMS, web chat, social, chat in IP messaging like WhatsApp, etc.

Think of vCon as ‘robot food’, enabling conversation data to be presented in a common format and more easily cleaned for training of machine learning. ASR and conversation AI solutions do not meet the needs of some businesses with respect to accuracy, vCon will help our industry close the gap with respect to the hype.

The performance of ASR varies greatly depending on the application, quality of the recording, and engine/training. ASR continues to improve, some of these applications could be a stretch for a legacy call center, however, for some scenarios they are attainable today.

Use Cases / Studies

vCons

Virtualized Conversations, or vCons, are an emerging standard designed to transform how organizations capture, store, and analyze human communication. Developed under the guidance of the Internet Engineering Task Force (IETF), vCons serve as structured, tamper-proof digital containers—akin to PDFs for conversations—that encapsulate metadata, transcripts, participant identities, AI-driven analyses, and related attachments. By standardizing conversational data into a consistent JSON-based format, vCons facilitate interoperability across platforms, enhance data integrity, and support compliance with privacy regulations like GDPR and CCPA.

The utility of vCons spans various industries, including customer service, healthcare, finance, and automotive sectors. For instance, in contact centers, vCons enable seamless integration between communication systems and analytical tools, reducing reliance on proprietary formats and simplifying data exchange. They also support advanced applications such as sentiment analysis, fraud detection, and personalized customer interactions by providing a unified framework for storing and processing conversational data.

Beyond operational efficiencies, vCons play a pivotal role in enhancing data governance and ethical AI deployment. By offering granular control over data access and retention, they empower organizations to uphold user rights and ensure transparency in automated decision-making processes. As conversational AI continues to evolve, the adoption of vCons is poised to become integral to responsible data management and the development of intelligent, user-centric communication systems.

For more information, you can explore the following resources:

vCons are...

A Brief Introduction to vCons and the Conserver

vCons are "PDFs" for Conversations

A vCon describes a conversation that involves a "natural" person. As a simple example, a vCon could be created from the last conversation you had with a customer service agent. Just like PDFs allow you to create and share any written document; vCons allow you to create and share any human conversation. A vCon identifies the people in the call, recordings and transcripts, analysis and supporting attachments, like documents and logs. Implemented in easy to manage JSON, vCons are both encryptable and tamper proof.

Conserver Introduction

A data platform for gathering, creating, storing and sharing vCons

The conserver is a data platform designed to extract conversations from business phone systems, transform them into actionable insights, and send that data into common business tools such as spreadsheets, Salesforce and no code toolsets. An open core product, the conserver enables data engineering teams to supply a reliable source of information for AI, ML and operational software in cloud, premise and hybrid contexts. The core for many of the business cases enabled by the conserver is the smart capture, redaction and lifecycle management of recorded customer conversations and customer journeys, recently accelerated by FTC and GDPR regulations and by increasing investments into AI and ML.

From a system perspective, shown above, the Conserver attaches to information systems like Web Chat and call center queues, and extracts information from them after conversations are ended. This information is then cleaned and transformed into actionable data. For instance, a distributed call center might extract conversations from a group of sales agents, convert them into text, then filter those conversations looking for times when customers objected to a sale. These objections are then pushed into database tables and Google Sheets as a data self-service option for any business team. The conserver supports multiple data pipelines, each one extracting data from a number of systems, performing transformations such as translations, transcriptions and redactions, and then pushing the prepared data into applications to be used.

In contrast to other data platforms, the Conserver is dedicated to managing the particular complexities of real time conversational sources. For instance, the amount of bandwidth and storage required to manage an hour long audio recording is an order of magnitude larger than managing a typical business object like a PDF. However, even this is just a start. Video is a few orders of magnitude greater than that, and the data creation for service providers such as Zoom and Skype are magnitudes of order still greater. From a legal perspective, regulatory compliance for customer data protections are particular for recorded conversations, and require support for tracking data’s use by automations, and for tracking deletion from a “Right to be Forgotten” request.

Why vCons?

vCons are a new data format that describe human conversations, allowing them to be secured and redacted, analyzed and tracked, storable and shareable.

Conversations are First Class Citizens

We have that enables anyone to store and exchange calendaring and scheduling information such as events, to-dos, journal entries, and free/busy information. And so anyone can store and exchange electronic business cards, name and address information, phone numbers, e-mail addresses, URLs (Universal Resource Locator), logos, photographs, and audio clips.

An innovation ecosystem of specialists in conversation intelligence will flourish solving specific business pain points across operations, compliance, privacy, security, ethics, etc. Being able to access customer data often trapped within communication platforms. Think of vCon as ‘robot food’, enabling conversation data to be presented in a common format and more easily cleaned for training of machine learning. ASR and conversation AI solutions do not meet the needs of some businesses with respect to accuracy; vCon will help our industry close the gap with respect to the hype.

More Information

Learning More

Official Documents

Read the

Track our

vCons are an Essential Tool in the Management of Personal Data

By defining the contents and participants of a conversation, vCons enable software tools that are able assert the presence, the absence and the authenticity of personal and biometric information. By tracking and consolidating the interactions with external systems with personal data, vCons enable responsible management of "Right to be Forgotten" regulations. vCons are best understood as a functional toolset for fine-grained control of personal information, authenticity, context and ethical treatment of personal data.

vCons are Open

vCons are both an open source project, it is also an open standards effort supported by the IETF. vCons enable the Internet to work better by defining the interchange standard for our most personal information: the sounds of of voice, and the images of our face. vCons enable the responsible management of personal data, enabling true compliance to customer data protections such as "Right to Know".

vCons are mostly After the Fact

Although you can build a vCon as a conversation happens, they are mostly understood to be an expression of a conversation that happened. vCons are primarily concerned with the responsible use of recorded conversations of all kinds. vCons are intended to be an effective tool for companies seeking GDPR compliance while handling recorded business conversations, which often contain sensitive biometric data.

vCons are the CDR Format for the age of AI

The conserver and vCons allows data controllers to be compliant in the age of AI:

  • vCons track the external systems, including AI, transcribers, redactors and analysis, enabling an authoritative answer to "What systems have seen my personal data?"

  • vCons and the conserver enable compliance to GDPR requirements, including "Right to Know", "Right to be Forgotten" and for responding to regulator audits for sensitive biometric information

  • vCons and the conserver allow you to share conversations across security boundaries, tracking shared data and requesting remote deletion.

IETF Virtualized Conversations Working Group
vCon GitHub Repository
A Comprehensive Guide to vCon in Communications

Conversational Data is Trapped in Silos

Despite all the talk about conversations in the programmable communications industry, conversation data remains trapped in silos. vCon (virtual Conversation, like vCard), is a new open standard for sharing conversation data: transcript, video, audio, participants, metadata like timestamps and location, tamper protections, certifications, etc. vCon has the potential to create an ecosystem of innovators focused on creating new conversation intelligence tools, in addition to established platform providers.

Sharing conversation data is a significant problem faced by programmable communications developers today. vCon makes working with conversations easier, hence the addressable market of developers expands ten thousand fold across web and enterprise developers. This happened one decade ago when telecoms became easy to use with simple web-centric APIs. This will happen again for conversations thanks to vCon.

vCon will do the same. Conversations currently trapped in silos of communication platforms or simply stored in the company’s data lake will become common units, where innovators will create new value, solve business problems, and deliver valuable insights because their business lives/dies on delivering that. vCon is the next leap forward in the programmable communications industry.

iCal
vCards
vCon Software and Implementations

Visit our open source library for the vCon and the Conserver. (Give us a star!)

The Repo for the py-vcon and py-vcon-server Projects

Videos and Presentations

See the Keynote at TAD Summit, Paris, October, 2023

See the TADSummit Podcast with Alan Quayle, September, 2023

See the Birds of a Feather session at IETF 116, Yokohama, March, 2023

See the presentation at TADSummit, Portugal, Nov 2022

See the presentation at IETF 115, London, Nov 2022

See the presentation at IIT, Chicago, Oct 2022

See the key note proposal for vCons.

White Papers

Read the white paper

IETF draft proposal
group's progress at the IETF

PII Compliance

The problems created by not maintaining PII compliance is more than negative publicity. The fines have been massive, for example the FTC (Federal Trade Commission) fined Facebook $5 billion in 2012, Equifax was fined at least $575 million in 2017 and 2019, and British Airways was fined $230 million in 2018. There’s a long list of regulations including: GDPR, HIPAA, CCPA, PCI DSS.

It’s not just the big brands that get caught, a PII audit can happen to any business. They are not scheduled, and can be triggered by a complaint that can come from an unhappy ex-customer or even a competitor. There are fines and possible incarceration for not reporting PII breaches as well.

PII can be a person's name, in combination with any of the following information:

• Mother's maiden name

• Driver's license number

• Bank account information

• Credit card information

• Relatives' names

• Home Postal address

• Personal E-mail address

• Home or cellular telephone number

• Personal characteristics / biometric data

• Social Security Number (SSN)

• Date or place of birth

• An individual's title

• Work telephone number

• Official work location/address

• Work email address

• Asset information, such as a car’s Vehicle Identification Number (VIN) or title number. Even MAC (Machine Address Code) or IMSI (International Mobile Subscriber Identity)

• Even a vCon record given voice finger-printing

• Other information that would make the individual's personal identity easily traceable

Across all the conversations a business has, both internally and externally, which are often recorded for training purposes or by company policy, there is a significant repository of customer data missed by most PII tools because they tend to focus on text based files.

Existing PII tools search the company’s storage for files containing customer data, e.g. usernames and password. vCon makes conversation data more easily available to the existing PII audit tools, so your business is better protected. Note, some PII audits are now warning of the emerging need of including audiovisual data for PII Compliance.

Through the open standard vCon the existing PII Compliance tools can be extended to files containing audio visual conversations that are often overlooked. A business is not trapped into multiple specialized PII audits across its different communication silos.

Customer Experience Improvement

Many of us have seen the data, 58% of customers say that customer service is a very important factor that affects their choice of a brand, source Microsoft’s State of Global Customer Service Report.

Call recording has been in place for decades, yet why have contact centers remained the same for decades? “Please listen carefully as our menus have changed.” No they have not, the menu has been the same for the past two decades! Why hasn’t call recording led to a virtuous circle of improvement?

Here are some of the claims made by call recording vendors:

  • Call center managers can review the calls to get a better understanding of how agents handle customer conversations.

  • Learn whether support representatives are following the protocols.

  • Figure out specific customer support aspects that can be improved.

  • With call recordings, call center managers can save time as they do not have to listen to each call in real time.

  • Identify the gaps in terms of training and best practices whether they are followed or not.

  • Learn first-hand customer feedback or issues and train your team to handle them better.

  • Listening to call recordings one on one with employees will empower managers to identify the skills that need to improve and work upon.

  • Based on the recorded audios, call center managers can prepare presentations to teach about the proper way of making and taking calls.

It’s rather manual and lacks quantified data and analysis, this reflects the historical limits of ASR. The call center manager will call a meeting based on their analysis, or bring in a consulting firm that analyzed the data, and based on ‘industry best practices’ make improvement recommendations. The training to implement the improvements is given, and when the call center manager evaluates the results, not much has changed. It’s been going on for decades. Often the sample size is too small given the highly manual approach, and sometimes changes to the process have unintended consequences blurring customer feedback.

vCon enables tens of thousands, even hundreds of thousands of calls to be analyzed by a broad ecosystem of innovative companies. Changes in the process can be analyzed by A/B analysis on possible ways to diffuse customer frustration about an overage charge. The vCons for the calls can be analyzed to make sure the A/B script is being followed, sentiment analysis, and customer feedback during / post call can be combined to produce quantified results.

Often great customer service is exemplified by employees going above and beyond what is economic for the business. While some companies like Zappos simply ensure a human approach to the customer. This may not work for all brands and situations. But being able to test and quantify the results over thousands or tens of thousands of conversations gives the business, and most importantly the agents, confidence in the process change. This is an example of a complex human factor problem that will require new approaches to conversation intelligence, enabled through the ecosystem made possible through vCon.

Insights from Customer Conversations

The customer is constantly providing information to your business: on which competitors they compare you to, what features they value in your product, what problems they have with your product, how they use your product, what features they do not use, how they like to be billed, how they view the pricing, why they moved from / to a competitor, how they use your product, how they mash-up your product and others to meet their needs, how they work around gaps in your product, etc.

Every conversation (voice, email, video, SMS, web chat, IP messaging, social, etc.) across sales, support, customer care, etc. is capturing this data. Extracting this data is not that easy, it's still a work in progress. vCon enables an ecosystem of analytics and data mining companies that can extract such information across all interactions. Determining that a customer is talking about a complementary product not a competitive product is not easy. But insights continue to improve.

vCon enables all customer data to be used to generate insights from the conversations stored but not fully mined. Surveys are often performed to capture minable data, often the surveys have leading questions or the survey is filled as fast as humanly possible. This does not provide accurate information, and why most surveys seem to enforce a marketing view not necessarily a customer view. The conversations taking place with sales, support, customer care are the most accurate customer data a business has access to, and remain relatively poorly mined.

Sharing Conversations with Third Parties

Working with contractors has its ups and downs. During meetings note taking can miss agreements on deliverables and their timing, simply not happen, or not be fully understood by all parties. Everyone leaves with a slightly different perception of the action. Missed deliverables become finger pointing exercises that do little to help the working relationship.

Most conference calls are recorded, yet are only used for those that missed the meeting and listened to at 1.5-2 X speed. A call recording is generally not used by the people who attended, because they already attended. However, the content in that call is invaluable. It’s not that we forget what was agreed on the call, it’s that we get busy, when the actions / deliverables are in black and white on our to-do list, we are reminded every day.

Imagine after the meeting a vCon is produced. The vCon is processed by a Meeting Minder - Construction Edition app, which sends a summary to all participants on the conference call that includes the actions and deliverables. The app is trained to the specific vocabulary and processes used in an industry vertical, construction.

Each action/deliverable includes a reference to the section in the conversation. For example, “Action: Plan for Floor 5 will be sent by Joe to Mary at Company X by end of day June 7th, spoken by Anne,<link to specific point in the conversation>”. There is no finger pointing, immediately after the meeting the summary was sent to everyone within the meeting minutes, and transferred onto each participant’s to-do list. If there was an error, it would have been picked up then.

Practically, ASR on a conference call is far from perfect, especially if someone calls in using a mobile phone. But the actions and summary will be checked by some of the participants to make sure everything is captured adequately.

vCon enables a range of value added services to be created around this open standard, an ecosystem of innovators taking conversation intelligence beyond transcription. No longer does the Meeting Minder app provider need to waste time and expense joining every conference call / collaboration platform’s partner ecosystem. With vCon they can implement one format and cover all communication platforms that use vCon. Instead focusing on building best in class industry vertical solutions.

The integration headache for the Meeting Minder app provider results in less competition. They will focus on the top 5 conference / collaboration platforms. The open source conferencing platforms, the regional platforms, and smaller providers are skipped. vCon helps maintain healthy competition and the reach of vCon apps to extend across all programmable communication platforms.

API

Introduction

Authorization

The Conserver API provides a token based authentication, controlled by the environment variable CONSERVER_API_TOKEN in the environment. When not defined or empty, it is disabled. To enable, define CONSERVER_API_TOKEN in the .env file

vCon Management

Chain Management

Chains are series of links that process a vCon. Before processing a vCon, be sure to load it.

Configuration

Dead Letter Queue

Lifecyle

Integrating Your App

Using the Conserver with your Application

Using the Conserver with your application is straightforward:

  • Setup your application database as a "storage" for the conserver. As it process vCons, the vCons will send copies of the vCons into the storage, similar to a database follower.

  • When you create, update and delete vCons, use the Conserver REST API. This assures that the vCons will be processed, tracked and protected like the rest.

  • For real time processing, you can use the use the WebHook link to notify your application of new vCons in the database. Also, since the conserver is a client to your application's database, all of the native notifications, such as Mongo OP-Log tails, REDIS key space events or S3 events can also be used.

Application Integration, Step by Step

  1. vCons are sent to the Conserver using the REST API to first create new vCons. These vCons will now be saved in the REDIS database using the vcon: keyspace. For instance, a vCon with a uuid of 018796f4-ece7-8f28-9dd8-dd37220d739c will be stored in REDIS JSON with a key of vcon:018796f4-ece7-8f28-9dd8-dd37220d739c.

  2. To process this vCon using a chain, add this vCon UUID to it using the REST API. These vCons uuid will be added to the list with matching name and processed in the main event loop.

  3. Once the chain has finished processing the vCon, it will put the vCon into the storages configured for that chain. In the diagram above, it will be stored in both S3 and MongoDB.

  1. Then using the REST API to trigger the execution of a "chain". Inside the conserver, the v

vCon Library

vCon Python Library

About the Library

The vCon (Virtual Conversation) library is a powerful Python tool designed to capture, structure, and manage conversation data in a standardized format. It provides a robust set of features for creating, manipulating, and analyzing digital representations of conversations, making it particularly useful for applications in customer service, call centers, chat systems, and any scenario where structured conversation data is valuable.

Concepts

Link

A link is the basic unit of processing in the conserver. A link takes a single vCon and processes it. Examples of links include:

  • Redaction: removes personal information from a transcript or a recording

Webhook: takes a vCon and HTTP Posts it to an external system
  • Summary: takes a transcript and summarizes the conversation

  • Stitcher: Takes information from external systems to improve vCon data. For instance. and inbound vCon could have the phone number of an agent. A stitcher will fill in missing details like name and email address.

  • A set of links forms a chain. A link can be a member of multiple chains. The conserver is responsible for delivering the vCon UUID to each link via a REDIS key. Links are described by REDIS keys with a prefix of link:, which are loaded on startup from the configuration file.

    Chain

    A chain is a series of links that processes a vCon. There is no theoretical limit to the number and length of chains. On a timer, the conserver iterates over each configured chain in the system. Chains are described by REDIS keys with a prefix of chain:, themselves loaded on startup from the configuration file.

    The input of the chain is one or more REDIS lists, which are themselves filled by adapters or by external systems, either directly in REDIS, or through the API.

    Storage

    A storage is an external data store, the "final resting place" for vCons as they are processed by the conserver, and are specified for each chain. Examples of storages include S3, Mongo, postgres or a local file system. For some storages, such as the file system. S3 or mongo, the storage format is the standard vCon format. For relational storages, such as postgres, a schema is defined in the repo.

    REDIS is used as a caching system, and vCons eventually expire from REDIS. When a REST API call refers to an expired vCon, it will be replaced in the cache.
  • Optionally, webhook links can be used at the end of a chain to notify external systems. Alternatively, applications can leverage the native notifications on their systems.

  • To find a vCon, use the application databases (in the figure shown as Mongo) native functionality. For instance, to find all vCons where a party is "[email protected]", the application would use the Mongo command: db.conversations.find_all({$.parties.email: "george.washington"}), assuming the vCons were kept in a collection called "conversations".

  • To maintain security, logging and synchronization of vCons, updates to the vCon should be made using the REST API. For instance, when a vCon is deleted using the REST API, it will also be deleted in any of the storages.

  • Application Integration with the Conserver

    At its core, the vCon library allows you to create vCon objects, which serve as containers for all elements of a conversation. These objects can include multiple parties (participants in the conversation), a series of dialogs (individual messages or utterances), metadata (such as tags for easy categorization), attachments (like transcripts or other related files), and even analysis data (such as sentiment analysis results).

    A vCon is the container for data and information relating to a real- time, human conversation. It is analogous to a [vCard] which enables the definition, interchange and storage of an individual's various points of contact. The data contained in a vCon may be derived from any multimedia session, traditional phone call, video conference, SMS or MMS message exchange, webchat or email thread. The data in the container relating to the conversation may include Call Detail Records (CDR), call meta data, participant identity information (e.g. STIR PASSporT), the actual conversational data exchanged (e.g. audio, video, text), realtime or post conversational analysis and attachments of files exchanged during the conversation. A standardized conversation container enables many applications, establishes a common method of storage and interchange, and supports identity, privacy and security efforts.

    Key capabilities of the vCon library include:

    1. Creating and managing vCon objects with a flexible, extensible structure.

    2. Adding and retrieving conversation participants (parties) with various attributes.

    3. Recording and organizing dialog entries with timestamps, content, and sender information.

    4. Attaching metadata and tags for easy categorization and searching.

    5. Including file attachments related to the conversation.

    6. Incorporating analysis data from various sources (e.g., sentiment analysis, topic classification).

    7. Signing and verifying vCon objects for data integrity and authenticity.

    8. Serializing vCon objects to and from JSON for easy storage and transmission.

    The library is designed with extensibility in mind, allowing for easy integration with various analysis tools and systems. It also includes built-in support for handling different types of conversation data, including text, audio, and video.

    By providing a standardized way to structure and manage conversation data, the vCon library enables powerful applications in areas such as conversation analytics, quality assurance, compliance monitoring, and machine learning model training for natural language processing tasks.

    Whether you're building a customer service platform, a conversation analysis tool, or any application that deals with structured dialog data, the vCon library offers a comprehensive solution for capturing, storing, and working with conversation information in a consistent and powerful way.

    Features

    • Create and manipulate vCon objects

    • Add parties, dialogs, attachments, and analysis to vCons

    • Sign and verify vCons using JWS (JSON Web Signature)

    • Generate UUID8 identifiers

    • Pack and unpack dialogs

    • Add and retrieve tags

    IETF vCon Working Group

    The vCon (Virtual Conversation) format is being developed as an open standard through the Internet Engineering Task Force (IETF). The vCon Working Group is focused on creating a standardized format for representing digital conversations across various platforms and use cases.

    Participating in the Working Group

    1. Join the Mailing List: Subscribe to the vCon working group mailing list at [email protected]

    2. Review Documents:

      • Working group documents and drafts can be found at: https://datatracker.ietf.org/wg/vcon/documents/

      • The current Internet-Draft can be found at: https://datatracker.ietf.org/doc/draft-ietf-vcon-vcon-container/

    3. Attend Meetings:

      • The working group meets virtually during IETF meetings

      • Meeting schedules and connection details are announced on the mailing list

      • Past meeting materials and recordings are available on the IETF datatracker

    4. Contribute:

      • Submit comments and suggestions on the mailing list

      • Propose changes through GitHub pull requests

      • Participate in working group discussions

    For more information about the IETF standardization process and how to participate, visit: https://www.ietf.org/about/participate/

    TADHack vCon

    Conversation Set

    For this year's TADHack vCon Hackathon, we've generated a set of synthetic vCons for your use:

    • You can download the set at https://github.com/vcon-dev/tadhack-2025

    • An S3 Bucket is here: arn:aws:s3:::tadhack-vcons

    Overview

    This dataset contains customer service conversation data from Aquidneck Yacht Brokers in VCON (Virtual Call Object Notation) format. The conversations span from May 18-24, 2025, and represent typical customer interactions for a yacht brokerage company. The dataset includes 42 customer service calls between Aquidneck Yacht Brokers agents and customers, covering various marine industry-specific support scenarios.

    Conversation Types

    1. Returns & Refunds

    • Customers requesting returns for yacht equipment

    • Processing refund requests

    • Emotional customers (often expressing sadness about returns)

    2. Shipping & Logistics

    • Yacht transportation inquiries (e.g., Fort Lauderdale to Newport)

    • Delivery status updates

    • Shipping cost questions

    3. Order Issues

    • Wrong items received (e.g., yacht anchor instead of navigation system)

    • Missing order investigations

    • Order verification and corrections

    4. Equipment Support

    • GPS malfunction troubleshooting

    • Navigation system issues

    • Equipment compatibility questions

    5. Business Services

    • Yacht listing inquiries

    • Brokerage service questions

    • Pricing and commission discussions

    6. Account Management

    • Membership cancellations

    • Billing inquiries

    • Privacy and data concerns

    • Contact information updates

    7. Appointments & Scheduling

    • Yacht viewing appointments

    • Service scheduling

    • Consultation bookings

    Call Characteristics

    • Average Duration: 50-60 seconds

    • Call Disposition: All marked as "ANSWERED" with "VM Left" status

    • Language: English

    • Transcription Confidence: 99%

    Data Format

    Each conversation includes:

    • Audio recording (MP3 format)

    • Full transcript with speaker diarization

    • AI-generated summary

    • Participant metadata (names, roles, contact info)

    Typical Interaction Flow

    1. Agent greeting with company name and agent introduction

    2. Customer name verification

    3. Issue description by customer

    4. Information gathering (order numbers, email verification)

    Notable Patterns

    • Customers frequently express emotions related to their issues

    • Agents consistently follow verification protocols

    • Marine industry-specific terminology used throughout

    • Focus on high-value transactions typical of yacht brokerage

    This dataset provides realistic examples of customer service interactions in the luxury marine industry, useful for training, analysis, or demonstration purposes.

    A vCon Primer

    Thomas McCarthy-Howe, CTO, Strolid.

    Introduction

    Responsible management of customer data was well understood, if not well distributed, before the AI revolution. Since the AI explosion set off by ChatGPT, the environment in which customer data must be protected is distinctly more hostile. Although difficult, you can change your name and your social security number. Changing the actual look of your face, or the actual sound of your voice, is near impossible. In a future filled with deep fakes, this is a problem demanding a solution for ethical system design, sound business operation and both commercial and civil governance.

    This primer explores the vCon, a groundbreaking technology designed to revolutionize the storage, analysis, and management of conversational data, of all kinds. This paper will provide a comprehensive overview of vCon, its structure, significance, and the stakeholders who should be interested in its implementation. For an current technical definition of a vCon, please visit the at the IETF (1).

    Help with implementations and interoperability testing

    Professional Tone: Agents maintain consistent, helpful demeanor

  • Resolution Rate: Most issues resolved or appropriately escalated

  • Call metadata (duration, timestamp, disposition)
    Resolution or escalation
  • Professional closing

  • What is vCon?

    A vCon (virtual conversation) is akin to a PDF, but instead of holding a readable document, it holds a recording of a conversation involving a “natural person” (2). It serves as a standardized format to document, store, and analyze conversations. The concept of vCon originated from a casual remark by Brian Galvin, a former CTO at Genesys, who mused about the lack of a vCard equivalent for conversations. Thus, the term vCon was coined: virtual conversations.

    Think of a vCon as a document format for conversations, ensuring that data is secure and authentic. The primary goal of a vCon is to enable a system of open tools to empower customer privacy and facilitate the management of personal information in conversations.

    Open Standard

    vCons, like other data formats such as Word and PDF, are open standards. By open, we mean they are publicly available and designed for use and implementation by anyone, facilitating transparency and compatibility across different systems. Like most patent offices, the United States Patent Office does not allow patents on data formats. The vCon is truly without any intellectual property encumbrances.

    This matches well with today’s data privacy challenges. Among the insidious threats of large language models is their opaque nature: unless revealed, the training data of an LLM is unknowable, thus the biases and intents of them are as well. vCons promote transparency by supporting an ecosystem of tools, applications and providers that exchange very sensitive data in a well known, and testable, format. vCons enables confident answers to “Is there personal data in this conversation?” and “Who created this vCon?”. This capability enables tools that can redact personal information, but also tools that can validate the same, each provided by otherwise independent developers.

    Why vCon Matters

    Privacy and Data Management

    Companies recording customer conversations inherently capture personal information, such as voices and faces, which are more sensitive than traditional identifiers like names. vCons help in managing and safeguarding this data, ensuring that companies can track, store, and delete data as required by regulations like GDPR.

    Compliance with Regulations

    vCons are designed to assist companies in complying with customer data regulations. For example, under GDPR, customers have the right to request deletion of their data. vCons provide a structured way to know what data was captured and ensure that it can be deleted or anonymized as required.

    Efficiency in Machine Learning

    Companies using customer data for machine learning need to manage this data responsibly. If a customer requests their data to be deleted, the company must retrain models without that data. vCons make it possible to track which models used which data, optimizing the retraining process and minimizing costs.

    Stakeholders Who Should Care

    Companies Recording Conversations

    Any company that records customer conversations needs to manage personal information responsibly. vCons provide a way to capture, store, and analyze this data while ensuring compliance with privacy regulations.

    Regulators Governing Personal Data Protection

    Companies that share customer data with other entities, such as automotive dealerships and manufacturers, need a reliable way to track and manage this information. vCons ensure that data can be traced and managed throughout its lifecycle.

    Companies Subject to Data Privacy Regulations

    Organizations operating in regions with strict data privacy laws, such as the EU under GDPR, must ensure that they can track and manage all captured data. vCons provide a structured way to meet these regulatory requirements.

    Companies Using Machine Learning

    Businesses leveraging machine learning models that use customer data need to be able to track and manage this data efficiently. vCons facilitate this by providing a clear record of what data was used, ensuring that models can be retrained as needed without excessive costs.

    Companies Managing Customer Consent

    Consent is hardly a static, nor boundless, idea. Consent is given for a purpose, with a time limit, and must be withdrawn upon request.

    Core Components of a vCon

    The Insides of a vCon

    Dialogues

    Dialogues form the heart of a vCon, encompassing all recorded media types, including text, messaging, video, and audio. vCons can be "packed" or "unpacked." A packed vCon includes media within the JSON package, making it suitable for scenarios where all parts need to be sent together, such as in an email. Unpacked vCons, on the other hand, are useful when large media files need to be managed separately from the core data package. Each dialog identifies a list of the parties in each dialog, directly enabling a customer’s “right to know”, as described in the GDPR and the CCPA.

    Parties

    Parties in a vCon identify conversation participants. It is crucial to note not only who was involved in the conversation but also who verified their identities.

    Analysis

    The analysis component involves commentary and insights derived from the dialogues. This can range from detecting emotions, recognizing significant events like birthdays, to identifying potential deceit in conversations. The analysis is stored as JSON arrays, making it easy to attach and manage various types of analytical data.

    Attachments

    Attachments provide context to the conversation. For example, a sales lead from Ford that prompted a call can be included as an attachment, enriching the context for future reference and analysis. This ensures that all relevant data is captured and can be used effectively by both humans and automated systems.

    Future Outlook

    The adoption of vCons is expected to grow as data privacy regulations become more stringent and the need for responsible data management increases. Companies will likely integrate vCons into their data engineering frameworks, ensuring that they can manage customer data effectively and comply with regulatory requirements.

    Conclusion

    vCon represents a significant advancement in the management of conversational data, offering a secure, standardized way to capture, store, and analyze conversations. By enabling better data management, vCons help companies comply with privacy regulations and optimize their use of customer data in machine learning applications. As the need for responsible data management grows, vCon is poised to become an essential tool for businesses worldwide.

    For more information, you can refer to the draft in the IETF, a white paper co-authored with Dan Petrie, and the working implementation of vCons in Python available on GitHub.

    Foot Notes

    1. https://datatracker.ietf.org/group/vcon/about/

    2. In the terminology of the GDPR, a natural person is an individual human being as opposed to a legal person such as a corporation.

    working group’s page

    MCP and AI

    How MCP Gives AI Assistants Real Capabilities

    AI assistants are powerful, but they have limits. They know a lot about the world from their training, but they cannot access your live data or use your tools. The Model Context Protocol, or MCP, changes that. This post explains how MCP works and why it matters.

    The Limitation of Training Data

    When you talk to an AI assistant like Claude or ChatGPT, it responds based on what it learned during training. That training data is a snapshot of information from when the model was created. It is like reading a book that was published last year. The information might be good, but it does not include anything that happened after publication.

    This creates several problems:

    • The assistant cannot see your current data

    • The assistant cannot perform actions in your systems

    • The assistant cannot access real-time information

    • The assistant cannot work with your specific tools and workflows

    You might ask the assistant to check your customer support calls from yesterday. Without MCP, the assistant cannot do that. It does not have access to your systems. It can only work with the information in its training data.

    What is Model Context Protocol?

    Model Context Protocol, or MCP, is an open standard that lets AI assistants interact with external tools and data sources. Think of it as a common language that AI assistants and your systems can both understand.

    MCP defines three main ways for assistants to interact with external systems:

    Tools - These are actions the assistant can perform. A tool might create a new record, search a database, or update information. Tools are like functions the assistant can call.

    Resources - These are read-only data sources the assistant can access. A resource might be a specific file, a database record, or a web page. Resources are like URLs the assistant can fetch.

    Prompts - These are templates that guide the assistant on how to do something. A prompt might explain how to search effectively or what information to include. Prompts are like instructions or recipes.

    Together, these three mechanisms give the assistant capabilities it did not have before.

    A Simple Analogy

    Imagine you have a smart assistant in your office. Without MCP, the assistant only knows what was in the training materials. It is like having someone who read a manual but has never actually used your office equipment.

    With MCP, you give the assistant:

    • Tools it can use, like your phone system, your database, and your file system

    • Resources it can read, like your customer records and your company documents

    • Prompts that explain how your office works, like how to file paperwork or who to contact for different issues

    Now the assistant can actually do work, not just answer questions about what it read in a manual. It can look things up, perform actions, and work with your actual systems.

    How MCP Extends AI Capabilities

    Let us look at each part of MCP in more detail.

    Tools: Actions the Assistant Can Perform

    Tools are executable operations. When you ask the assistant to do something, it can choose a tool that performs that action. For example, if you ask the assistant to find customer support calls about billing, it might use a search tool provided by your conversation database.

    Each tool has:

    • A name that describes what it does

    • A description that explains when to use it

    • Input parameters that define what information it needs

    • Output that describes what it returns

    The assistant understands these tool definitions and can decide when to use each one. It is like giving the assistant a toolbox where each tool is labeled and has instructions.

    Resources: Data the Assistant Can Read

    Resources are URI-based data access points. The assistant can request a resource and get data back. For example, if you ask about a specific conversation, the assistant might fetch a resource that points to that conversation's data.

    Resources are read-only. The assistant cannot change data through resources. It can only read. This makes resources safe for the assistant to explore your data without accidentally modifying anything.

    Resources are discoverable. The assistant can ask what resources are available and then access them. It is like giving the assistant a catalog of all the data it can read.

    Prompts: Guidance for the Assistant

    Prompts are templates that help the assistant understand how to accomplish tasks. They provide context and step-by-step guidance. For example, a prompt might explain how to search for conversations by tags, or what information to include when creating a new conversation record.

    Prompts are not just instructions. They can include examples, best practices, and warnings about common mistakes. They help the assistant work more effectively with your specific systems and requirements.

    How the vCon MCP Server Uses MCP

    The vCon MCP Server implements MCP to give AI assistants access to conversation data. It provides:

    Over 27 tools for managing conversations. These include creating conversations, searching them, updating them, adding analysis, managing tags, and getting analytics.

    Resources that let the assistant directly read conversation data using URI paths. For example, the assistant can request a resource like vcon://uuid/abc123 to get a specific conversation, or vcon://uuid/abc123/parties to get just the participant information.

    9 prompts that guide the assistant on effective searching and retrieval. These prompts help the assistant understand when to use different search strategies and how to structure queries.

    When you ask the assistant to work with conversation data, it uses these tools, resources, and prompts. The assistant does not need to know the technical details of how your database works. It just needs to know which tools to use and how to use them.

    An Example Conversation

    Let us see how this works in practice. You are talking to your AI assistant:

    You: "Find all the customer support calls from last week where the customer was frustrated."

    Assistant: The assistant sees you want to search for conversations. It looks at the available tools and sees several search options. It decides to use semantic search because you mentioned "frustrated," which is about sentiment rather than exact words. The assistant uses the search_vcons_semantic tool with parameters like the date range and a query about customer frustration.

    Assistant: "I found 12 support calls from last week where customers showed frustration. Would you like me to summarize the common issues, or show you specific calls?"

    You: "What were the main issues?"

    Assistant: The assistant uses the search results it already has and analyzes them. It might use additional tools to get more details about specific conversations, or it might analyze the data directly from the search results.

    Assistant: "The main issues were: delivery delays (5 calls), billing errors (4 calls), and product defects (3 calls). Should I get more details on any of these?"

    This conversation flows naturally, but behind the scenes the assistant is using MCP tools to access your actual conversation data. It is not making things up based on training data. It is working with your real data.

    Why Standards Matter

    MCP is an open standard, not a proprietary system. This means:

    • Any AI assistant can implement MCP support

    • Any system can provide MCP tools, resources, and prompts

    • You are not locked into one vendor

    • The community can improve and extend the standard

    This is important because it means MCP will work with future AI assistants, not just the ones available today. If a new assistant comes along that you prefer, it can still use the same MCP servers you have set up.

    It also means you can build your own MCP servers for your specific needs. You are not limited to what vendors provide. You can create tools that match exactly what your business needs.

    Benefits of the MCP Approach

    Using MCP with AI assistants provides several benefits:

    Real-time access - The assistant can work with your current data, not just historical training data.

    Actionable capabilities - The assistant can perform actions, not just answer questions.

    System integration - The assistant can work with your existing tools and databases.

    Natural interaction - You talk to the assistant in plain language, and it figures out which tools to use.

    Extensibility - You can add new tools, resources, and prompts as your needs grow.

    Security - The assistant only has access to what you explicitly provide through MCP. You control what it can see and do.

    How It Differs from Traditional APIs

    You might wonder how MCP differs from traditional APIs. Traditional APIs require you to know specific endpoints, parameters, and response formats. You need to write code or configure integrations.

    MCP works at a higher level. The assistant understands what tools are available and how to use them. You do not need to write code or configure complex integrations. You just talk to the assistant, and it handles the details.

    This does not mean MCP replaces APIs. MCP often uses APIs under the hood. But it presents them to the assistant in a way the assistant can understand and use intelligently.

    The Future of AI Integration

    MCP represents a new way of integrating AI assistants into your work. Instead of treating the assistant like a separate tool, MCP lets you treat it like a team member who has access to your systems.

    As MCP grows and more systems adopt it, AI assistants will become more capable. They will be able to work with more types of data and perform more types of actions. The vCon MCP Server is one example of this future. It gives assistants the ability to work with conversation data in a standard way.

    Conclusion

    MCP bridges the gap between AI assistants and your systems. It gives assistants real capabilities by providing tools, resources, and prompts they can understand and use. The vCon MCP Server implements MCP to make conversation data accessible to AI assistants.

    The next post in this series covers the complete scope of what the vCon MCP Server can do. It goes into detail about all the features and capabilities available.

    What is the vCon MCP Server?

    Have you ever wanted to ask an AI assistant about past conversations? Maybe you want to find all the times a customer called about billing issues, or analyze patterns in support calls, or track what happened in a sales meeting. The vCon MCP Server makes this possible.

    This post explains what the vCon MCP Server is, what problem it solves, and why it might be useful for you.

    The Problem with Conversation Data

    Most businesses have conversations happening everywhere. Phone calls, video meetings, chat messages, emails. These conversations contain valuable information, but they are usually scattered across different systems. Each system stores data in its own format. This makes it hard to:

    • Search across different types of conversations

    • Analyze patterns over time

    • Share conversation data between tools

    • Work with AI assistants on conversation history

    • Maintain privacy and compliance standards

    You might have customer support calls in one system, sales meetings in another, and email threads in yet another. To get a complete picture, you would need to check all three systems separately. That takes time and effort.

    What is vCon?

    vCon stands for Virtual Conversation. It is an IETF standard format for representing conversations. Think of it like PDF for conversations. Just as PDF is a standard format that works across different computers and programs, vCon is a standard format that works across different systems.

    A vCon file can contain:

    • The actual conversation content, whether it came from voice, video, text, or email

    • Information about who participated in the conversation

    • Analysis results from AI, like transcripts, sentiment scores, or summaries

    • Attachments like documents or images related to the conversation

    The key benefit is portability. If you store conversations in vCon format, you can move them between systems without losing data. You are not locked into one vendor's system. You own your conversation data in a standard format.

    What is MCP?

    MCP stands for Model Context Protocol. It is a way for AI assistants to use external tools and data sources. Without MCP, AI assistants can only work with the information they learned during training. They cannot access your live data or perform actions in your systems.

    With MCP, an AI assistant can:

    • Read data from your databases

    • Perform actions using your tools

    • Access real-time information

    • Maintain context about what you are working on

    Think of MCP like giving an AI assistant access to your toolbox. The assistant can see what tools are available, understand what each tool does, and use them when you ask. This makes AI assistants much more useful for real work.

    What is the vCon MCP Server?

    The vCon MCP Server combines these two ideas. It is a server that lets AI assistants work with conversation data stored in vCon format. You connect the server to an AI assistant like Claude, and then the assistant can:

    • Create new conversation records

    • Search through historical conversations

    • Analyze conversations for insights

    • Organize conversations with tags

    The server speaks the MCP protocol, which AI assistants understand. When you ask the assistant to do something with conversation data, it uses the server's tools to get the job done.

    What Can It Do?

    Here are the main capabilities:

    Store conversations - The server can store conversations in vCon format, following the IETF standard exactly.

    Search conversations - You can search in four different ways:

    • Basic filtering by subject, participants, or dates

    • Keyword search that looks for exact words

    • Semantic search that finds conversations by meaning, even if the exact words are different

    • Hybrid search that combines keyword and semantic approaches

    Organize with tags - You can add tags to conversations for easy organization and filtering. Tags work like labels you might put on file folders.

    Analyze and monitor - The server can provide analytics about your conversation database, showing growth trends, content patterns, and health metrics.

    Manage components - You can add or update different parts of a conversation, like adding analysis results or attaching files, without recreating the whole conversation.

    Use templates - The server includes templates for common conversation types, making it easier to create new records.

    Extend with plugins - The server supports plugins that can add custom functionality, like privacy controls or compliance features.

    Who Would Use This?

    Several groups of people might find this useful:

    Customer support teams - Store and search support calls, track issues, analyze agent performance, and maintain compliance records.

    Sales teams - Record sales conversations, extract action items, analyze what works, and generate meeting summaries.

    Compliance and legal teams - Maintain conversation archives, apply privacy controls, track consent, and generate audit reports.

    Researchers - Collect conversation datasets, study communication patterns, and build training data for machine learning models.

    Developers - Build applications that work with conversation data using a standard format and API.

    Business analysts - Search across conversations to find insights, track trends, and answer questions about customer interactions.

    A Simple Example

    Imagine you run a customer support team. You have thousands of support calls stored in a system. You want to know: "What are customers complaining about most this month?"

    Without the vCon MCP Server, you might need to:

    1. Export data from your phone system

    2. Load it into a spreadsheet or database

    3. Write queries or scripts to analyze it

    4. Create reports manually

    With the vCon MCP Server, you can simply ask your AI assistant: "What are customers complaining about most this month?" The assistant uses the server's search tools to find relevant conversations, analyzes them, and gives you an answer. If you want more detail, you can ask follow-up questions. The assistant has access to all your conversation data through the server.

    Why Standards Matter

    Both vCon and MCP are open standards. This means:

    • They are not controlled by a single company

    • Anyone can implement them

    • They work across different systems

    • They evolve through community input

    Using standards gives you options. If you build on top of the vCon MCP Server and later want to switch to a different system, your data is in a standard format. You are not locked in. You also benefit from the work others do with these standards. New tools and integrations appear as the standards grow.

    Getting Started

    The vCon MCP Server is open source and free to use. You need:

    • Node.js installed on your computer

    • A Supabase account for the database (free tier available)

    • An AI assistant that supports MCP, like Claude Desktop

    The server connects to your database and exposes tools that the AI assistant can use. You talk to the assistant in natural language, and it figures out which tools to use and how to use them.

    What's Next?

    This was a high-level overview. If you want to learn more, the next posts in this series cover:

    • How MCP works with AI assistants in more detail

    • The complete scope of what the server can do

    • How the server is built and why it is designed that way

    • Real-world business cases and use cases

    Each post goes deeper into different aspects of the server. You can read them in order or jump to what interests you most.

    Configuring the Conserver

    A Complete Guide

    The conserver's configuration file is the heart of how the system operates, defining how conversations flow through the system and how they are processed and stored. Let's break down each major component and how to configure them.

    Configuration File Location

    The configuration file location is specified in the environment (I use the .env file), typically at config.yml in the vcon-server root.

    Configuration File Structure

    The configuration file is a YAML document with several main sections:

    • links: Defines the processing modules available to the system

    • storages: Specifies where vCons can be stored

    • chains: Defines the workflow pipelines

    • followers: Configures how the system can follow other conservers

    Let's explore each section in detail:

    Configuring Links

    Links are the processing units of the conserver. Each link is a module that performs a specific operation on a vCon. Here's how to configure a link:

    Each link configuration needs:

    • A unique name (e.g., 'deepgram', 'analyze')

    • The module path that implements the link functionality

    • An options dictionary containing the link's specific configuration

    Configuring Storages

    Storages define where vCons are saved after processing. The conserver supports multiple storage backends:

    Each storage needs:

    • A unique name

    • The storage module implementation

    • Connection and authentication options specific to the storage type

    Configuring Chains

    Chains are where you define your processing workflows. They connect links together and specify where the results should be stored:

    A chain configuration includes:

    • The links to execute, in order

    • Input lists (ingress_lists) where new vCons arrive

    • Storage locations for the processed vCons

    • Output lists (egress_lists) for downstream processing

    Configuring Followers

    Followers allow one conserver to monitor and process vCons from another conserver:

    Each follower needs:

    • The URL of the remote conserver

    • Authentication credentials

    • The remote list to monitor (egress_list)

    • The local list to populate (follower_ingress_list)

    Configuration Best Practices

    When configuring your conserver:

    1. Use meaningful names for your chains, links, and storage configurations to make the system easier to understand and maintain.

    2. Consider your processing pipeline carefully - organize links in a logical order where each step builds on the previous ones.

    3. Use multiple storage backends when needed - for example, storing in both S3 for long-term storage and Postgres for quick querying.

    4. Configure appropriate timeouts for your chains based on the expected processing time of your links.

    The configuration file is loaded by the system at startup and can be updated via the API endpoint /config. The system will use the new configuration immediately after updating.

    Remember that the conserver uses Redis as its working storage, so all the lists referenced in ingress_lists and egress_lists are Redis lists. The actual vCons are stored in Redis using JSON data types, making them quickly accessible during processing.

    This configuration system provides a flexible way to define complex processing pipelines for your vCons while keeping the configuration clear and maintainable.

    Inside the Conserver

    The Machinery of the Conserver

    The System view of the Conserver

    The Conserver processes vCons, storing them locally, and projecting them into the third party information services. The building blocks of the Conserver are links, which have an interface to accept a single vCon, and can then forward that vCon, or create new ones, to other links for further processing. Links are formed into chains, designed to apply a series of analysis and transformation to the vCons. Chains are executed by the conserver periodically on a timer, or on request from a third party system.

    Links: The Fundamental Building Block

    The heart of the conserver functionality is the "link". A link is a Python module that takes a single vCon and processes it. Chains are ultimately created by combining links in serial. All links have the same interface. Using links has multiple advantages:

    • Configurable: Uses a flexible options system for customization.

    • Retry Mechanism: Implements exponential backoff for API call retries.

    • Caching: Avoids redundant analysis by checking existing data.

    • Metrics: Tracks performance and error metrics.

    As an example, let's look at This link takes a vCon, applies a prompt to it, then adds an analysis to the vCon with the result.

    Main Function: run

    The run function is the entry point for links. , it performs the following steps:

    1. Merges provided options in the "config.yml" file with default options.

    2. Retrieves the vCon (voice conversation) object from Redis.

    3. Applies inclusion filters and sampling.

    4. Iterates through dialog entries in the vCon:

    Default Options

    A default_options dictionary defines the options for the link, and are overridden by the configuration file. For instance, the analysis link is defined with the following options:

    • Prompt for summarization

    • The value to set as the analysis type when added to the vCon (default: "summary")

    • GPT model (default: "gpt-3.5-turbo-16k")

    • Sampling rate and temperature

    Error Handling and Metrics

    The module includes error handling for API calls and retries. It also tracks metrics such as analysis time and failures using custom metric functions.

    Return Values

    Links can return one of two kinds of values. Links can return a vCon UUID, or None. Typically, it would be the vCon UUID that was passed in. However, if the link created a new vCon, as would be required for creating a new, redacted vCon, the new UUID would be returned by the link. To stop chain processing, a link could return None. This is useful for links that filter vCons out, only allowing certain ones down the chain, and stopping the processing of links downstream of the chain.

    Chains: Links for Workflow

    The fundamental implementation of workflow is created by a series of links. These chains take vCon uuids from REDIS lists, runs the chain of links on the vCon, stores it, then places the uuids in egress REDIS links.

    Chain Processing, Link by Link

    The processes vCons:

    • Loads configuration and sets up the ingress chain map.

    • Enters a loop that continuously checks for new items in the ingress lists using Redis.

    • When an item (vCon ID) is found, it creates a VconChainRequest and processes it.

    • Handles exceptions by moving problematic vCons to a Dead Letter Queue.

    Step by step:

    1. Processing starts when vCon UUIDs are placed into a ingress list. Chains may have several ingress lists, and have to have at least one to kick off processing. Lists are implemented as REDIS lists, and processing is controlled at the thread layer by blocking until the a new element is placed on the list. UUIDs can be added to the ingress list by other chains, allowing them to be placed in series, from links that can request processing, or from the API. A typical pattern is to create the vCon using the API, then inserting the UUID into the desired ingress list.

    2. For each vCon taken from the ingress list, it is processed by each link in the chain. This _process_link function handles the execution of a single link in the processing chain for a vCon. Here's a summary of its functionality:

      This enables flexible and dynamic execution of different processing steps (links) in the vCon processing chain, with built-in logging and timing measurements.

    Tech Stack

    The Conserver is built off of two core platforms: a python API framework FASTAPI, and a REDIS real time database. The conserver itself is written in Python, and uses the standard vCon Python library to create and modify vCons.

    REDIS is responsible for storing the conversations, while FAST API coordinates the application software that manages them. Each conversation is stored as a REDIS JSON object in the standard vCon format. In practice, each vCon is stored in REDIS by the UUID of the vCon, making them easy to discover and fast to process. Instead of copying the conversation as it’s built and transformed, it stays stored in REDIS, and the ID to the vCon is passed, optimizing processing efficiency even at very large data sizes. REDIS also provides inter task communication using a series of PUB/SUB channels, coordinating the activities of the conserver for both local software (that inside the conserver itself) but also for external software such as Lambdas or exporting onto other systems like Apache Kafka. Also, third party and hardware enabled systems can use REDIS as a data interchange system, loading and unloading large media files in coordination with the data pipeline.

    Each vcon is stored in REDIS using JSON and named with a regular key: vcon:{{vcon-uuid}}, as are chains "chains:{{name}}", links "link:{{name}}" and storages "storage:{{name}}}". REDIS allows for the addition of dedicated hardware to accelerate long running and high compute use cases such as transcription and video redaction, as these systems can connect directly to REDIS relieving scale issues from general purpose hardware, while managing the overhead of moving large amounts of data. Links take a vCon ID as inputs, and bear the responsibility of reading vCons if required, or giving them the option to hand off to optimized hardware.

    FAST API provides the application infrastructure for the conserver. The transformation steps are developed as Python modules and loaded as tasks managed by FAST API. As each task finishes, it notifies other system elements by publishing UUID of the vCon. Other tasks wait on these notifications, and when they receive the notification, they can act on that same vCon for whatever purpose they may have. In addition, FAST API provides a REST API to the store of vCons, and a simple UI to manage the conserver.

    vCon Apps and Stores

    The vCon Store: Building an Open Ecosystem for Conversational Applications

    Introduction: From Infrastructure to Innovation

    The history of telecommunications reveals a profound truth: what we perceive as infrastructure today often began as revolutionary applications. Alexander Graham Bell's telephone wasn't conceived as the backbone of global communication—it was commissioned by Samuel Gridley Howe, president of the Perkins School for the Blind, as an assistive device for the blind, built atop existing telegraph infrastructure. This pattern of application-driven innovation has repeated throughout telecommunications history, from Interactive Voice Response (IVR) systems that businesses once had to be convinced to adopt, to prepaid calling cards that drove massive traffic in the telecommunications field during the 1970s and 80s, each representing an attempt to unlock new value from voice communications.

    GitHub - vcon-dev/vcon: The Home Repo for vConsGitHub
    Our Open Source Repository for vCons and the Conserver
    Virtualized Conversations (vcon)datatracker.ietf.org
    The IETF vCon Group

    Documentation is public and accessible

    Privacy markers that track consent and can hide sensitive information

    Answer questions about your conversation data
    They are documented publicly

    Modular: Designed to be part of a larger system, likely for processing voice conversations.

    Retrieves the source text for analysis.

  • Checks if analysis already exists.

  • Generates new analysis using OpenAI if needed.

  • Adds the generated analysis to the vCon object.

  • Stores the updated vCon back in Redis.

  • Source configuration for transcript analysis, for instance "transcript" or "summary"

    It logs the start of processing for the specific link and vCon.

  • It retrieves the link configuration from the global config.

  • It dynamically imports the module specified for this link if it hasn't been imported before, caching it for future use.

  • It retrieves any options specified for the link.

  • It logs the execution of the link's module.

  • It measures the execution time of the module's run method, which is called with the vCon ID, link name, and options.

  • After execution, it logs the completion of the link processing, including the time taken.

  • Finally, it returns the result from the module's run method, which determines whether the chain should continue processing or stop.

  • After the links have been processed, assuming that none of the links returned None, the vCon UUID is pushed into the chain's egress lists. Finally, the vCon is then stored in the storages (S3, Mongo, File, etc.) specified for that link.

  • In case of an error in any of these links, the vCon UUID will be pushed into the dead letter queue of the original ingress list.

  • the analyze link.
    For the analysis link
    main loop of the conserver
    A Conserver Link
    A Chain

    An enabled flag and optional timeout

    Polling configuration (interval and batch size)

    Use the follower configuration when you need to process vCons across multiple conserver instances, creating distributed processing pipelines.

    links:
      deepgram:
        module: links.deepgram
        options:
          DEEPGRAM_KEY: your_key_here
          minimum_duration: 30
          api:
            model: "nova-2"
            smart_format: true
            detect_language: true
    
      analyze:
        module: links.analyze
        options:
          OPENAI_API_KEY: your_key_here
          prompt: "Summarize this transcript"
          analysis_type: summary
          model: 'gpt-4'
    storages:
      postgres:
        module: storage.postgres
        options:
          user: postgres
          password: your_password
          host: your_host
          port: "5432"
          database: postgres
    
      s3:
        module: storage.s3
        options:
          aws_access_key_id: your_key_id
          aws_secret_access_key: your_secret
          aws_bucket: your_bucket
    chains:
      transcription_chain:
        links:
          - deepgram
          - analyze
          - webhook_store
        ingress_lists:
          - transcription_input
        storages:
          - postgres
          - s3
        egress_lists:
          - transcription_output
        enabled: 1
        timeout: 300
    followers:
      remote_conserver:
        url: "https://remote-conserver.example.com"
        auth_token: "your_auth_token"
        egress_list: "remote_output"
        follower_ingress_list: "local_input"
        pulling_interval: 60
        fetch_vcon_limit: 10

    Even the early SIP phone initiatives, despite sophisticated technical foundations, struggled to find compelling applications that resonated with users. The iPhone represented what many thought would finally deliver true phone applications, but the reality proved different—the iPhone succeeded not as a phone with applications, but as a handheld computer where the phone function became just one capability among many. Notably, until very recently, most applications on smartphones weren't truly "phone applications" that leveraged voice communication as their core functionality.

    Today, we stand at a similar inflection point with vCons (virtual Conversation records) and the emerging ecosystem of conversational applications. What we're seeing from the vCon application community is genuinely stunning—sophisticated dashboards, business intelligence tools, and AI-powered insights that were previously impossible. The question isn't whether voice applications will evolve—it's how we can architect systems that enable innovation rather than constrain it.

    Understanding vCons: A PDF for Conversations

    Before exploring the store architecture, it's essential to understand what makes vCons uniquely powerful. A vCon functions as "a PDF for conversations"—a standardized, tamper-proof, signed, and encryptable JSON document that captures the complete context of any communication interaction. Each vCon contains four key components:

    Parties - The identities of all participants in the conversation, providing clear attribution and context for every interaction.

    Dialogs - The actual content of what was communicated, whether through voice, video, or text, preserving the full conversational record.

    Analysis - Automated insights from AI and machine learning systems that track sentiment, extract key topics, identify action items, and provide business intelligence.

    Attachments - Supporting and associated data including documents, images, or any files relevant to the conversation.

    This structure creates a complete, portable record of human communication that can be processed, analyzed, and acted upon by applications while maintaining data integrity and authenticity.

    The Application Evolution: Why Documents Trump APIs

    The telecommunications industry has consistently struggled with the tension between innovation and integration complexity. While early attempts at voice applications often failed due to architectural limitations, today's vCon-based applications are demonstrating remarkable success. The key insight driving this success comes from years of sponsoring vCon mashup competitions with the TadHack community: vCon mashups are becoming dominant not just because vCons are cool (though they are), but because a document is easier than an API.

    This principle explains why vCon applications are gaining traction where previous voice application attempts failed. When developers can base applications on standardized document formats rather than navigating complex API integrations, the barrier to entry drops dramatically. More people can participate, tools can more easily process the data, and innovation accelerates. The fact that vCons exist as files rather than API endpoints represents what might be called "accidental magic"—a design decision that unlocks unprecedented accessibility for application developers.

    The file-based nature of vCons means integration becomes as simple as "file URL to file"—a pattern that's easy to test, easy to describe, and works seamlessly across different systems and platforms.

    Architectural Foundation: The Model-View-Controller for Conversations

    Modern application development relies heavily on the Model-View-Controller (MVC) architecture, which separates concerns into three distinct layers: the data model, the business logic controller, and the presentation view. This separation allows developers to create applications that work consistently across mobile, web, and messaging interfaces while maintaining a single source of truth for data and logic.

    The vCon ecosystem maps naturally onto this proven architecture. Instead of traditional database models, conversations become the foundational data layer. The same controller patterns that manage business logic and access controls apply seamlessly to conversational data. Views can range from dashboards and reports to AI-powered interfaces, all drawing from the same conversational foundation.

    This architectural alignment isn't coincidental—it represents a maturation of conversational technology that makes it compatible with standard development practices and tools.

    Advanced Architecture: The Conserver System

    The technical implementation of vCon hosting involves sophisticated architecture that balances performance, security, and accessibility. The Conserver system represents the middleware layer that makes the vCon store ecosystem possible, handling everything from real-time processing to long-term storage and consent management.

    The Conserver architecture employs a processing pipeline with multiple specialized components. Incoming vCons flow through analysis, transcription, and large language model processing before being distributed to various storage systems. High-speed access through Redis enables real-time applications, while long-term storage in systems like S3 provides cost-effective archival. The system also supports webhook notifications for real-time application updates and integrates with various AI services through standardized interfaces.

    Model Control Protocol (MCP) integration represents a particularly innovative aspect of the architecture. As MCP emerges as a standard for AI system integration, vCon hosters can provide MCP interfaces as a standard connection point for their customers' existing AI ecosystems. Whether integrating with OpenAI, Claude, Watson X, or other AI platforms, the MCP interface provides a consistent integration pattern that eliminates the need for custom API development.

    The system also includes SCITT (Supply Chain Integrity, Transparency and Trust) compatible ledger capabilities for enhanced security and auditability. This blockchain-inspired approach ensures that vCon records maintain integrity and provide verifiable audit trails—critical for regulatory compliance and trust verification in sensitive business communications.

    For consent management, the architecture implements comprehensive privacy controls with real-time enforcement. When a data subject revokes consent, the system can immediately propagate deletion requests through all storage layers, from high-speed caches to long-term archives, ensuring compliance with privacy regulations like GDPR.

    The vCon Store: A Four-Component Ecosystem

    The proposed vCon store architecture consists of four essential components, each serving a distinct role while maintaining clear boundaries and responsibilities:

    vCon Creators form the foundation, encompassing all systems that generate conversational records—phone systems, email platforms, chat applications, voice automation systems, and emerging AI agents. Legacy equipment can participate through vCon adapters, while native voice suppliers and modern communication systems can generate vCons directly. The diversity of creator types ensures that virtually any communication platform can participate in the ecosystem.

    vCon Hosters serve as the custodians of conversational data, operating as data controllers responsible for storage, protection, and access management. These entities bear the crucial responsibility of data rights protection, consent management, and compliance with privacy regulations. By centralizing these concerns with specialized providers, the architecture allows other ecosystem participants to focus on their core competencies without becoming privacy law experts.

    The hosting function includes sophisticated data management capabilities: enterprise-grade databases, cloud services integration, real-time processing capabilities, and comprehensive audit trails. Hosters can integrate with existing business systems through webhooks, APIs, and emerging standards like MCP (Model Control Protocol).

    Data Subjects retain ultimate control over their conversational data through managed consent mechanisms. The architecture ensures clear accountability—data subjects know exactly who has access to their conversations and why, with straightforward mechanisms for consent withdrawal that cascade through all system components.

    vCon-Enabled Applications represent the innovation layer, where developers create value-added services without needing to worry about data storage, privacy compliance, or integration complexity. These applications connect to data stores populated by Conservers using familiar database interfaces and development patterns. Application categories include voice analytics dashboards, compliance monitoring systems, customer intelligence platforms, fraud detection tools, and custom enterprise applications tailored to specific business needs.

    Breaking Down Walled Gardens

    The current communications landscape suffers from what Mark Twain might recognize as history rhyming with itself. Just as AOL once controlled users' access to news, weather, and sports by being the single gateway to information, today's communication platforms often capture conversations, analyze them, and control application access within closed ecosystems.

    The vCon store architecture offers an alternative path—one that separates concerns and prevents any single entity from controlling the entire value chain. In this open ecosystem, hosters can choose their level of openness, service providers can focus on core capabilities without managing hundreds of API integrations, and application developers can reach users through standardized interfaces rather than platform-specific implementations.

    This separation creates a truly competitive marketplace where each component can excel at its specific function while participating in a larger, interoperable ecosystem.

    Practical Implementation: From Theory to Reality

    Real-world implementations demonstrate the practical viability of this architecture, with applications being developed in remarkably short timeframes. Enterprise applications using vCons stored in Snowflake and accessed through Python Jupyter notebooks can be developed in minutes rather than months. The "vCon Quality Report" dashboard—complete with quality metrics, conversation analytics, and even a patron saint of quality data (Saint Vincenzo)—was actually created as a joke during a team standup meeting, yet provides genuine business value by tracking vCon creation rates, summarization progress, and system performance.

    The quality report shows practical metrics like daily vCon generation (6,752 vCons on the day measured, down 20% because it was Sunday), summarization rates approaching 100%, and detailed analytics on conversation duration and patterns. This level of business intelligence, traditionally requiring extensive custom development, becomes straightforward when working with vCons as standardized data files.

    Similarly, small business applications like conversational diaries can be built using standard tools: MongoDB for storage, OpenAI for processing insights, and Streamlit for user interfaces. These applications provide immediate value by summarizing daily conversations, extracting action items, and identifying business opportunities. For example, a simple diary application can show "what happened today," list actionable next steps, and highlight potential opportunities—all derived automatically from the day's conversational data.

    The BMW dealership example showcases how conversational intelligence can transform business operations, providing detailed summaries of customer interactions, agent performance metrics, and actionable insights for improving service delivery. The system tracked 168 calls on a single day, breaking down agent performance and identifying specific customer needs and opportunities.

    The Strategic Imperative

    The vision for vCon stores extends beyond technical architecture to market strategy. By focusing on file-based standards rather than proprietary APIs, the ecosystem can support much wider participation and generate better results for all participants. Hosters can focus on their core competency of secure, compliant data management. Application developers can create innovative solutions without becoming integration specialists. End users benefit from choice, interoperability, and innovation.

    This approach promises to unlock the same kind of explosive growth that occurred when the internet broke down AOL's walled garden, giving users access to unlimited sources of information and services rather than a single provider's curated selection.

    Conclusion: Files as the Foundation of Innovation

    The telecommunications industry has repeatedly demonstrated that breakthrough applications drive infrastructure evolution, not the reverse. Today's vCon ecosystem represents the latest iteration of this pattern, with document-based architectures enabling a new generation of conversational applications.

    The proposed vCon store architecture offers a path forward that balances innovation with responsibility, openness with security, and simplicity with capability. By treating conversations as files and building standard architectures around them, we can create an ecosystem where innovation flourishes while protecting the rights and interests of all participants.

    The future of conversational applications lies not in more complex APIs or tighter platform integration, but in simpler, more open architectures that let developers focus on creating value rather than managing complexity. The vCon store represents a concrete step toward that future—one file at a time.

    Conserver Quick Start

    A quick start to getting the conserver up and running

    Ubuntu Install

    Based on a digital ocean install, to keep it vanilla. Created a 4 GB Memory / 2 Intel vCPUs / 120 GB Disk / NYC3 - Ubuntu 23.04 x64 droplet, logged in.

    Conserver Start

    The conserver repo , but is also included in the vcon repo in the von-server directory.

    Create an ~/vcon/.env file. See example .env below. *Note that the default URL for REDIS assumes it is running in a docker container, thus the hostname "redis".

    Example vcon-server/config.yml

    Most of the configuration is done through the config.yml file. Here's a very simple one. Inbound vCons in the ingress chain cause a slack webhook into a workflow, then it will be stored in mongo.

    Standalone Operation

    When running a conserver in "standalone mode" (using vcon-admin as a simple portal, which will also provide the basic versions of all of the apps and databases), it will automatically register a domain name and generate a valid SSL certificate using LetsEncrypt, assuming that the domain name has an A record pointing to your server.

    Start the Conserver

    Troubleshooting and Checking

    You can validate that the conserver is running on the command line using "docker ps". In the example below, we can see four instances running.

    You can see the operational logs using "docker compose logs -f". Here's a typical log:

    The is a nice tool for managing the conserver.

    SCITT: Supply Chain Integrity, Transparency and Trust

    A Framework for Securing Modern Software Supply Chains

    Abstract

    The increasing complexity of software supply chains has created unprecedented security challenges, as demonstrated by high-profile attacks like SolarWinds and Log4Shell. Supply Chain Integrity, Transparency and Trust (SCITT) emerges as a comprehensive framework designed to create an immutable, transparent ledger for software supply chain artifacts and attestations. This whitepaper examines SCITT's architecture, security model, and interoperability features, demonstrating how it addresses critical gaps in current supply chain security approaches while maintaining compatibility with existing tools and workflows.

    1. Introduction

    Modern software development relies on complex supply chains involving multiple parties, from open-source contributors to commercial vendors. Each component may pass through numerous hands before reaching production systems, creating opportunities for compromise at every stage. Traditional approaches to supply chain security rely on point-in-time verification and trust relationships that can be exploited by sophisticated attackers.

    SCITT provides a standardized framework for creating tamper-evident, publicly verifiable records of claims about software artifacts throughout their lifecycle. By establishing a cryptographically secured, append-only registry of attestations, SCITT enables organizations to make verifiable statements about their software while allowing consumers to independently verify these claims and trace complete component histories.

    2. Technical Architecture

    2.1 Core Components

    SCITT's architecture consists of four primary components:

    Transparency Service: The central registry that maintains an append-only log of all claims. This service provides cryptographic receipts proving that claims have been registered at specific points in time.

    Claims: Signed statements about software artifacts, including but not limited to Software Bills of Materials (SBOMs), vulnerability reports, build provenance, and security assessments. Claims are format-agnostic, allowing integration with existing standards.

    Receipts: Cryptographic proofs issued by the Transparency Service confirming that a claim has been registered. These receipts enable offline verification without accessing the service.

    Verifiers: Entities that validate claims and their associated receipts to establish trust in software components before deployment or use.

    2.2 Cryptographic Foundation

    SCITT builds upon established cryptographic standards, primarily CBOR Object Signing and Encryption (COSE) for signature formats. This ensures compatibility with existing public key infrastructure while providing flexibility for future cryptographic algorithms. The framework uses hash chains similar to certificate transparency logs, creating an immutable record that makes tampering immediately detectable.

    3. Relationship to Blockchain Technology

    While SCITT shares fundamental concepts with blockchain technology, it represents a specialized application optimized for software supply chain metadata rather than general-purpose transactions.

    3.1 Shared Principles

    Both SCITT and blockchain systems implement:

    • Immutable append-only logs where historical records cannot be altered

    • Cryptographic linking between entries to ensure temporal ordering

    • Transparent verification allowing any party to audit the complete history

    • Decentralization potential through federated architectures

    3.2 Key Differentiators

    Unlike traditional blockchains, SCITT:

    • Eliminates consensus overhead: No proof-of-work or proof-of-stake mechanisms required

    • Enables efficient scaling: Designated transparency services operated by trusted organizations

    • Separates storage and verification: Federated registries can reference each other without full replication

    • Optimizes for specific use cases: Designed specifically for software attestations rather than financial transactions

    This specialized approach makes SCITT more energy-efficient and performant while maintaining the security guarantees essential for supply chain integrity.

    4. Security Benefits

    4.1 Non-Repudiation and Accountability

    Once an organization registers a claim about their software, the cryptographic receipt creates an undeniable record. This accountability mechanism ensures that malicious actors cannot quietly inject compromised components or retroactively alter their attestations after a breach is discovered.

    4.2 Tamper-Evidence and Forensic Capabilities

    SCITT's cryptographic receipts provide temporal proof of when specific claims existed, enabling precise forensic analysis. Security teams can definitively determine what was known at any point in time, identifying exactly when and where compromises occurred in the supply chain.

    4.3 Attack Prevention and Detection

    SCITT addresses several critical attack vectors:

    Supply Chain Injection: Unauthorized modifications become immediately visible due to missing or invalid SCITT registrations from legitimate sources.

    Dependency Confusion: Internal packages can be cryptographically distinguished from public packages through issuer verification.

    Time-of-Check/Time-of-Use: Cryptographic receipts ensure the verified version matches the deployed version.

    Retroactive Tampering: The append-only nature prevents attackers from covering their tracks by modifying historical records.

    4.4 Policy Enforcement

    Organizations can implement automated security policies that verify multiple conditions before allowing software deployment:

    • Presence of required attestations (SBOMs, vulnerability scans)

    • Signatures from authorized entities

    • Build provenance from approved CI/CD systems

    • Compliance with regulatory requirements

    These policies can be enforced programmatically across organizational boundaries, creating a transparent trust framework.

    5. Interoperability and Integration

    5.1 Format-Agnostic Design

    SCITT's architecture accepts any serializable content type, enabling seamless integration with existing tools and standards:

    • SPDX and CycloneDX for software composition

    • In-toto and SLSA for build provenance

    • Custom formats for proprietary security assessments

    • Industry-specific compliance attestations

    This flexibility allows organizations to adopt SCITT without abandoning their current toolchains or workflows.

    5.2 Federation Capabilities

    Multiple SCITT instances can interoperate through claim references, enabling:

    • Cross-organizational trust without centralization

    • Industry-specific registries that maintain autonomy

    • Geographic distribution for performance and compliance

    • Gradual adoption across supply chain participants

    5.3 Standardized APIs

    SCITT employs standard HTTP REST APIs with COSE signatures, ensuring:

    • Language-agnostic integration

    • Minimal modification to existing tools

    • Consistent verification regardless of claim format

    • Simplified adoption across diverse ecosystems

    6. Implementation Considerations

    6.1 Deployment Models

    Organizations can choose from several deployment approaches:

    Public Registries: Industry-wide transparency services operated by trusted entities Private Registries: Internal services for proprietary software and sensitive attestations Hybrid Models: Selective publication based on confidentiality requirements Federated Networks: Interconnected registries sharing trust relationships

    6.2 Performance and Scalability

    SCITT's design prioritizes efficiency:

    • Lightweight claim registration process

    • Offline verification capabilities

    • Distributed caching of receipts

    • Selective synchronization between registries

    These characteristics enable SCITT to scale to global software supply chains without creating performance bottlenecks.

    6.3 Migration Strategies

    Organizations can adopt SCITT incrementally:

    1. Begin with high-value or high-risk components

    2. Integrate with existing CI/CD pipelines

    3. Gradually expand coverage across the software portfolio

    4. Establish federation with supply chain partners

    7. Future Directions

    The SCITT framework continues to evolve through the IETF standardization process. Key areas of development include:

    • Enhanced privacy features for sensitive attestations

    • Improved federation protocols for cross-registry trust

    • Integration with emerging software identity standards

    • Automated policy languages for complex trust requirements

    8. Conclusion

    SCITT represents a fundamental advancement in software supply chain security, providing the transparency and accountability necessary for modern software ecosystems. By combining the immutability of blockchain-inspired architectures with the efficiency required for practical deployment, SCITT offers a path toward comprehensive supply chain integrity.

    The framework's format-agnostic design and standardized APIs ensure that organizations can adopt SCITT without disrupting existing workflows, while its cryptographic foundation provides the security guarantees necessary to detect and prevent sophisticated supply chain attacks. As software supply chains continue to grow in complexity, SCITT's transparent trust model becomes increasingly critical for maintaining security at scale.

    References

    • IETF SCITT Working Group. "Supply Chain Integrity, Transparency and Trust." Internet Engineering Task Force. https://datatracker.ietf.org/wg/scitt/about/

    • SCITT Architecture Internet-Draft. "An Architecture for Supply Chain Integrity, Transparency, and Trust." https://datatracker.ietf.org/doc/draft-ietf-scitt-architecture/

    • Microsoft Corporation. "SCITT Confidential Consortium Framework Ledger Implementation." https://github.com/microsoft/scitt-ccf-ledger

    • SCITT Community. "SCITT API Emulator Reference Implementation." https://github.com/scitt-community/scitt-api-emulator

    Day In the Life of a vCon

    To illustrate the normal operation of the conserver, let’s follow along as a conversation is extracted, transformed and the data is provided to a business team. For this example, we’ll assume the Conserver is started and configured to take conversations from a Freeswitch system, transcribe them, look for a particular subject (recalls) and send those to a PostGres table for the operations team.

    1. A customer and an agent has a conversation using Freeswitch. A Freeswitch adapter is running that monitors calls and requesting recordings. For context, refer to https://developer.signalwire.com/compatibility-api/xml/ to see the kinds of call events and recording options.

    2. When the call on Freeswitch ends, the adapter uses the data from the call (parties, recordings) to create a vCon. This vCon is then sent to the Conserver in a POST to the conserver's API, naming the REDIS lists that feed each conserver chain. Alternatively, the vCon could also be inserted into REDIS directly, then adding the vCon UUID to each chain's ingress list.

    snap install docker
    git clone https://github.com/vcon-dev/vcon.git
    cd vcon/
    git submodule sync
    git submodule update --init --recursive
    cd vcon-server/
    git checkout main
    cd ..
    cd vcon-admin/
    git checkout main
    cd ..
  • Internet Engineering Task Force. "CBOR Object Signing and Encryption (COSE)." RFC 8152. https://datatracker.ietf.org/doc/html/rfc8152

  • Cybersecurity and Infrastructure Security Agency. "Software Supply Chain Security Guidance." https://www.cisa.gov/software-supply-chain-security

  • SCITT Receipts Format Specification. "SCITT Receipts." https://datatracker.ietf.org/doc/draft-ietf-scitt-receipts/

  • Package URL Specification. "A minimal specification for purl." https://github.com/package-url/purl-spec

  • Supply-chain Levels for Software Artifacts. "SLSA Framework." https://slsa.dev/

  • OpenSSF Sigstore Project. "A new standard for signing, verifying and protecting software." https://www.sigstore.dev/

  • This vCon is stored in REDIS as a JSON object under the UUID of the newly created vCon. By convention, the key is named with the pattern “vcon:uuid” (like vcon:7665-343535-58575-333).

  • In addition to the standard parts of a vCon, the dialog and parties, the adapter adds a new attachment (to the attachments section of the vCon standard) that details what adapter created the vCon, details important for debugging, etc. This attachment travels inside the vCon throughout it’s life, unless it is explicitly stripped off later on.

  • Based on a periodic timer, or triggered by an external API call, the conserver iterates over all of the processing chains. Each chain has a REDIS list that contains the vCons to be processed. On each tick, the conserver creates a task for each ID that is read from the list. Horizontal scaling is enabled by having a single REDIS cluster connected to multiple conservers. Each task iterates the vCon over the series of links in the chain.

  • In this example chain, the first link is called “transcription”, and unsurprisingly, transcribes conversations. Links expect a vCon UUID as an input, and return vCon UUIDs as outputs. This allows configurations of chains of links, the output of one feeding the input of the next, freely interchangeable in order, or vendor.

  • The transcription link (currently there are two versions to choose from, Whisper.ai and Deepgram) take the dialog section of the vCon (which holds the recorded voice) and transcribe them. This transcription is added to the vCon in the “analysis” section, and normally contains information like a complete transcription, and a confidence score and a time stamp for every word transcribed. The link then updates the stored vCon with this new analysis, using REDIS to avoid reading or copying the large data objects in the dialog.

  • The conserver is responsible for the ordering and execution of each link in the chain. It is not a requirement that a link be used once; it may be repeated several times within and between chains.

  • The second link in the chain is called “recall finder”, and uses the output of the transcription link. When it is called by the conserver, it loads the transcription attachment and looks for the word “recall” in the conversation. If it does not find the word, it can simply exit without creating any message for the downstream plugin, effectively ending the processing of that chain of links.

  • At this point, the vCon has been created, captured, transcribed and identified as having the information we want: it’s a recall conversation. For information systems that want a native JSON representative, the vCon can now be sent for consumption. For instance, it could now be sent via a web hook (HTTP POST) to any API endpoint. In like manner, it can be stored in any of the storage blocks, current options include the Mongo Database, REDIS, S3, PostGres or a local file system.

  • If the final destination has a fixed schema, like a Postgres database, a Google Spreadsheet or a no code tool, we need to create a “projection” for this data before the “recall finder” is done. A projection is a simple key-value store of the important information, determined by the use case. For illustration, assume we are interested in sending the transcription, the identity of the agent and the customer, and when the conversation happened. This projection, which directly corresponds to a single row in a database, with four columns (transcription, agent, customer, created at), will be added to the vCon, just as the transcription analysis was. At this point, the original vCon now has an attachment from the adapter, an analysis by the transcriber, and this new transcription analysis.

  • The final link is a PostGres projection. When it runs, it looks for projections on a vCon, then takes that information and uses it to add (or upsert) a new row with the information from the projection into a configured PostGres table. From the perspective of the business users of the data, they simply see rows of transcribed conversations that match a criterion. Data projections, like adapters, handle the differences between destinations: unique data projections are required for different kinds of relational database, no code tools, Google Sheets, etc.

  • Speech Recognition Test Set

    ASR (Automatic Speech Recognition) for many languages is still in development. That is why Le Voice Lab exists, a French association that brings together various institutional players (universities, research laboratories, etc.) and private companies whose common interest is to build an independent ecosystem and common standards to enable France and Europe to remain competitive in the global voice market. It’s not just Europe, around the world there is a substantial quality gap compared to the English speaking world for ASR.

    But which ASR works best for business’s customers, and is it good enough for the intended applications? Enterprises are now equipped to easily compare the different ASR engines from global and regional providers. vCon enables a single source of test data to accurately and repeatedly measure speech recognition performance across 100s or 1000s of samples, to gather statistically meaningful performance data.

    As it’s a computer file format the vCons can be processed through an Excel sheet or business intelligence application. Businesses can make quantified decisions based on their specific situation. The ‘Rolls Royce’ ASR may be the best with an accuracy range of 94-96%, but the ‘Honda Civic’ ASR is good enough at 92-94% for the intended application. The vCons from the different ASRs can be processed through the business application, and the business results compared, not just word error rates.

    A business may receive 95% of their voice calls from 3G mobile networks with a range of dialects. They can build their own vCon test set, run them through the ASRs, and with nothing more than Excel compare the results. It could be that ASR in general is not currently up to the task, this will change given the continued performance improvements, but better to make an informed decision and revisit; than assume ASR is inadequate until the gap with competitors becomes clear and leaves your business struggling to catch up.

    vCons democratize an opaque industry, which relies on fear, uncertainty, and doubt to stop the buyer making a quantified decision that is best for their situation.

    can be downloaded directly
    vCon admin program
    cd vcon-server
    REDIS_URL=redis://redis
    
    # Leave this blank to disable API security
    # You set this before opening the port in your firewall
    CONSERVER_API_TOKEN=
    
    # to customize the config copy example_config.yml to config.yml
    # modify the values in config.yml as needed
    # and set CONSERVER_CONFIG_FILE to ./config.yml below
    CONSERVER_CONFIG_FILE=config.yml
    links:
      tag:
        module: links.tag
        ingress-lists: []
        egress-lists: []
        options:
          tags:
          - smart_trunk_ingress
      
    storages:
      mongo:
        module: storage.mongo
        options:
          url: mongodb://root:example@mongo:27017/
          database: conserver
          collection: vcons
    chains:
      demo_chain:
        ingress_lists:
        - ingress
        links:
        - tag
        storages:
        - mongo
        enabled: 1
    export [email protected]
    export DNS_HOST=mulligan.strolid.net
    docker network create conserver
    docker compose build
    docker compose up
    docker compose up --scale conserver=4 -d
    root@partner-demo:~/vcon/vcon-server# docker ps
    CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS                   PORTS                                                 NAMES
    21bc6e3aacd7   vcon-server-conserver      "/app/docker/wait_fo…"   4 minutes ago   Up 4 minutes                                                                   vcon-server-conserver-4
    2e3a0341043d   vcon-server-conserver      "/app/docker/wait_fo…"   4 minutes ago   Up 4 minutes                                                                   vcon-server-conserver-2
    9c699287f035   vcon-server-conserver      "/app/docker/wait_fo…"   4 minutes ago   Up 4 minutes                                                                   vcon-server-conserver-3
    ffe6f68941c8   vcon-server-conserver      "/app/docker/wait_fo…"   5 minutes ago   Up 5 minutes                                                                   vcon-server-conserver-1
    8136e15912c5   vcon-server-api            "/app/docker/wait_fo…"   5 minutes ago   Up 5 minutes             0.0.0.0:8000->8000/tcp, :::8000->8000/tcp             vcon-server-api-1
    e3388b5f23be   redis/redis-stack:latest   "/entrypoint.sh"         5 minutes ago   Up 5 minutes (healthy)   6379/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp   vcon-server-redis-1
    root@partner-demo:~/vcon/vcon-server# 
    vcon-server-redis-1      | 9:C 23 Aug 2024 17:27:20.581 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    vcon-server-redis-1      | 9:C 23 Aug 2024 17:27:20.582 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    vcon-server-redis-1      | 9:C 23 Aug 2024 17:27:20.582 * Redis version=7.4.0, bits=64, commit=00000000, modified=0, pid=9, just started
    vcon-server-redis-1      | 9:C 23 Aug 2024 17:27:20.582 * Configuration loaded
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.582 * Increased maximum number of open files to 10032 (it was originally set to 1024).
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.583 * monotonic clock: POSIX clock_gettime
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.584 * Running mode=standalone, port=6379.
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.586 * Module 'RedisCompat' loaded from /opt/redis-stack/lib/rediscompat.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.614 * <search> Redis version found by RedisSearch : 7.4.0 - oss
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.616 * <search> RediSearch version 2.10.5 (Git=2.10-e2f28a9)
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.616 * <search> Low level api version 1 initialized successfully
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.617 * <search> gc: ON, prefix min length: 2, min word length to stem: 4, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results:  10000, 
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.620 * <search> Initialized thread pools!
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.621 * <search> Enabled role change notification
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.621 * Module 'search' loaded from /opt/redis-stack/lib/redisearch.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.630 * <timeseries> RedisTimeSeries version 11202, git_sha=5643fd4d6fcb1e9cf084fb2deb9285b08f4a6672
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> Redis version found by RedisTimeSeries : 7.4.0 - oss
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> loaded default CHUNK_SIZE_BYTES policy: 4096
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> loaded server DUPLICATE_POLICY: block
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> loaded default IGNORE_MAX_TIME_DIFF: 0
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> loaded default IGNORE_MAX_VAL_DIFF: 0.000000
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> Setting default series ENCODING to: compressed
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * <timeseries> Detected redis oss
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.631 * Module 'timeseries' loaded from /opt/redis-stack/lib/redistimeseries.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Created new data type 'ReJSON-RL'
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> version: 20803 git sha: unknown branch: unknown
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Exported RedisJSON_V1 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Exported RedisJSON_V2 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Exported RedisJSON_V3 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Exported RedisJSON_V4 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Exported RedisJSON_V5 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <ReJSON> Enabled diskless replication
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * Module 'ReJSON' loaded from /opt/redis-stack/lib/rejson.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.639 * <search> Acquired RedisJSON_V5 API
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.641 * <bf> RedisBloom version 2.8.2 (Git=unknown)
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.641 * Module 'bf' loaded from /opt/redis-stack/lib/redisbloom.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.648 * <redisgears_2> Created new data type 'GearsType'
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.650 * <redisgears_2> Detected redis oss
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.652 # <redisgears_2> could not initialize RedisAI_InitError
    vcon-server-redis-1      | 
    vcon-server-redis-1      | 
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.652 * <redisgears_2> Failed loading RedisAI API.
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.652 * <redisgears_2> RedisGears v2.0.20, sha='9b737886bf825fe29ddc2f8da81f73cbe0b4e858', build_type='release', built_for='Linux-ubuntu22.04.x86_64', redis_version:'7.4.0', enterprise:'false'.
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.657 * <redisgears_2> Registered backend: js.
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.657 * Module 'redisgears_2' loaded from /opt/redis-stack/lib/redisgears.so
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.657 * Server initialized
    vcon-server-redis-1      | 9:M 23 Aug 2024 17:27:20.657 * Ready to accept connections tcp
    vcon-server-conserver-2  | Redis is ready!
    vcon-server-conserver-2  | Redis is ready. Starting the dependent service...
    vcon-server-conserver-2  | {"asctime": "2024-08-23 17:28:24,696", "levelname": "INFO", "name": "__main__", "message": "Starting main loop", "taskName": null}
    vcon-server-conserver-4  | Redis is ready!
    vcon-server-conserver-4  | Redis is ready. Starting the dependent service...
    vcon-server-conserver-4  | {"asctime": "2024-08-23 17:28:24,545", "levelname": "INFO", "name": "__main__", "message": "Starting main loop", "taskName": null}
    vcon-server-conserver-3  | Redis is ready!
    vcon-server-conserver-3  | Redis is ready. Starting the dependent service...
    vcon-server-conserver-3  | {"asctime": "2024-08-23 17:28:25,041", "levelname": "INFO", "name": "__main__", "message": "Starting main loop", "taskName": null}
    vcon-server-api-1        | Redis is ready!
    vcon-server-api-1        | Redis is ready. Starting the dependent service...
    vcon-server-api-1        | Skipping virtualenv creation, as specified in config file.
    vcon-server-api-1        | {"asctime": "2024-08-23 17:27:24,198", "levelname": "INFO", "name": "server.api", "message": "Api starting up", "taskName": "Task-1"}
    vcon-server-api-1        | {"asctime": "2024-08-23 17:27:24,226", "levelname": "INFO", "name": "uvicorn.error", "message": "Started server process [1]", "taskName": "Task-1", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}
    vcon-server-api-1        | {"asctime": "2024-08-23 17:27:24,226", "levelname": "INFO", "name": "uvicorn.error", "message": "Waiting for application startup.", "taskName": "Task-1"}
    vcon-server-api-1        | {"asctime": "2024-08-23 17:27:24,227", "levelname": "INFO", "name": "uvicorn.error", "message": "Application startup complete.", "taskName": "Task-1"}
    vcon-server-api-1        | {"asctime": "2024-08-23 17:27:24,227", "levelname": "INFO", "name": "uvicorn.error", "message": "Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)", "taskName": "Task-1", "color_message": "Uvicorn running on \u001b[1m%s://%s:%d\u001b[0m (Press CTRL+C to quit)"}
    vcon-server-conserver-1  | Redis is ready!
    vcon-server-conserver-1  | Redis is ready. Starting the dependent service...
    vcon-server-conserver-1  | {"asctime": "2024-08-23 17:27:22,240", "levelname": "INFO", "name": "__main__", "message": "Starting main loop", "taskName": null}
    

    Business Cases for MCP Servers and vCon

    This post explains why conversation data matters in business, what problems organizations face with it, and how MCP servers using the vCon format can help. It also covers specific use cases and considerations for implementation.

    Why Conversation Data Matters

    Businesses have conversations with customers, partners, and employees every day. These conversations contain valuable information that can drive better decisions and outcomes.

    Customer insights - Conversations reveal what customers care about, what problems they face, and how they feel about your products or services. This information is more direct and actionable than surveys or analytics.

    Operational intelligence - Conversations show how well your team is performing, what processes work, and where improvements are needed. You can identify training opportunities, process bottlenecks, and quality issues.

    Compliance requirements - Many industries must maintain records of certain conversations for regulatory compliance. Healthcare, finance, and legal services all have specific requirements.

    Relationship management - Conversation history helps teams understand relationships over time. When a customer calls, agents can see past interactions and provide better service.

    Analytics and research - Aggregated conversation data can reveal trends, patterns, and insights that inform strategy, product development, and marketing.

    Despite this value, many organizations struggle to capture, organize, and use conversation data effectively.

    Problems with Conversation Data

    Organizations face several common problems when working with conversation data:

    Data fragmentation - Conversations happen across many systems. Phone calls are in one system, emails in another, chat messages in yet another, and video meetings somewhere else. Getting a complete picture requires checking multiple systems.

    Vendor lock-in - Each system stores data in its own format. Moving data between systems is difficult, and switching vendors often means losing access to historical data or spending significant effort to migrate it.

    Limited search capabilities - Most systems only support basic search by date, participant, or subject. Finding conversations by meaning, sentiment, or topic is difficult or impossible.

    Integration challenges - Connecting conversation data to other business systems requires custom development. Each integration is a new project with ongoing maintenance costs.

    Compliance complexity - Meeting regulatory requirements often means building custom solutions for consent tracking, data retention, redaction, and audit trails.

    AI integration difficulty - AI assistants and analysis tools cannot easily access conversation data because it is locked in proprietary systems with limited APIs.

    These problems make it hard to realize the full value of conversation data.

    How MCP Servers Solve These Problems

    MCP servers provide a standard way for AI assistants to access conversation data. This solves several problems:

    Standardized access - AI assistants understand the MCP protocol, so they can work with any MCP server without custom integration code. You can switch AI assistants or use multiple ones without rebuilding integrations.

    Natural language interaction - Instead of writing code or learning specific APIs, you can ask AI assistants to work with your conversation data using plain language.

    Real-time capabilities - MCP servers provide access to live data, not just historical snapshots. AI assistants can work with current information.

    Extensibility - MCP servers can expose multiple tools, making them more capable over time without requiring changes to how assistants interact with them.

    The Value of the vCon Format

    The vCon format adds additional value on top of MCP:

    Portability - vCon is a standard format that works across systems. You can move conversation data between vendors or tools without losing information. You own your data in a format you can use anywhere.

    Completeness - vCon captures all aspects of a conversation in one place. Participants, content, analysis results, attachments, and metadata are all included. You get a complete picture, not fragments scattered across systems.

    Interoperability - Because vCon is a standard, different tools can work with the same data. A transcription service, an analysis tool, and a compliance system can all use the same vCon files.

    Future-proofing - As new tools and services emerge, they can work with your vCon data because it follows a standard. You are not locked into today's technology choices.

    Compliance-ready - The vCon format includes fields for consent tracking, privacy markers, and redaction. You can implement compliance features on top of standard data.

    Detailed Use Cases

    Here are specific examples of how organizations use MCP servers with vCon for business value:

    Contact Centers

    Contact centers handle large volumes of customer interactions. They need to track issues, analyze performance, and maintain compliance.

    How they use it: The center stores all customer calls in vCon format through an MCP server. Agents and managers can ask AI assistants questions like "What are the top three issues customers called about this month?" or "Show me calls where customers were frustrated."

    Value delivered:

    • Faster issue identification and resolution

    • Better training based on actual customer interactions

    • Automated compliance reporting

    • Performance analytics without manual data collection

    Specific capabilities:

    • Search for calls by topic, sentiment, or issue type

    • Generate summaries of common problems

    • Track resolution rates and customer satisfaction

    • Identify training opportunities for agents

    Sales Teams

    Sales teams have conversations with prospects and customers throughout the sales process. These conversations contain information about needs, objections, timelines, and decision criteria.

    How they use it: Sales teams record calls and meetings in vCon format. They tag conversations with deal stages, product names, and customer segments. They can ask AI assistants to extract action items, identify decision makers, or find similar past deals.

    Value delivered:

    • Better deal tracking and forecasting

    • Faster onboarding for new sales team members

    • Insights into what messaging works

    • Automatic extraction of next steps and commitments

    Specific capabilities:

    • Search for conversations about specific products or features

    • Find similar past deals to inform strategy

    • Extract action items and follow-up tasks automatically

    • Analyze which approaches lead to closed deals

    Compliance and Legal Teams

    Legal and compliance teams must maintain records of regulated communications and ensure they meet retention, privacy, and audit requirements.

    How they use it: Teams store all regulated communications in vCon format. Plugins add compliance features like automatic retention policy enforcement, consent tracking, and audit logging.

    Value delivered:

    • Automated compliance with regulations like GDPR, CCPA, and HIPAA

    • Complete audit trails for regulatory reviews

    • Efficient response to privacy requests

    • Reduced risk of compliance violations

    Specific capabilities:

    • Automatic redaction of sensitive information

    • Consent tracking and management

    • Retention policy enforcement

    • Audit log generation for regulators

    Research and Analytics Teams

    Research teams collect conversation datasets to study communication patterns, build training data for machine learning, or analyze language use.

    How they use it: Teams store research conversations in vCon format, which provides a standard structure for analysis. They can export data in a format that works with analysis tools.

    Value delivered:

    • Standardized data format across research projects

    • Easy integration with analysis tools

    • Reproducible research with consistent data structures

    • Sharing datasets with other researchers

    Specific capabilities:

    • Export conversations in standard formats

    • Search and filter conversations by research criteria

    • Anonymize data for sharing

    • Track metadata about research participants

    Healthcare Organizations

    Healthcare organizations must document patient interactions while maintaining strict privacy and compliance standards.

    How they use it: Organizations store patient consultation records in vCon format. Plugins add HIPAA compliance features, access controls, and integration with electronic health record systems.

    Value delivered:

    • Better documentation of patient interactions

    • Compliance with healthcare regulations

    • Integration with existing health record systems

    • Improved care coordination through accessible records

    Specific capabilities:

    • Store consultation transcripts and notes

    • Link conversations to patient records

    • Automatic privacy controls and access restrictions

    • Generate clinical summaries automatically

    ROI Considerations

    Implementing an MCP server with vCon format can provide returns in several areas:

    Time savings - Teams spend less time searching for information, exporting data, or switching between systems. AI assistants can answer questions that previously required manual work.

    Better decisions - Access to conversation insights leads to better decisions about products, services, processes, and strategy.

    Compliance cost reduction - Automated compliance features reduce the manual work and risk associated with meeting regulatory requirements.

    Integration cost reduction - Standard formats reduce the cost of integrating conversation data with other systems. Instead of custom integrations for each system, you use standard formats.

    Vendor flexibility - Not being locked into a single vendor allows you to choose tools based on what they do best, not just what integrates with your existing system.

    Future capability - As new AI tools and services emerge, you can take advantage of them because your data is in a standard format.

    When to Use vCon MCP Server vs Alternatives

    The vCon MCP Server is a good choice when:

    • You need to work with conversation data across multiple systems or tools

    • You want AI assistants to access your conversation data

    • You value data portability and avoiding vendor lock-in

    • You need to meet compliance requirements with conversation data

    Alternatives might be better when:

    • You only use a single conversation system and do not need integration

    • Your conversation volume is very small and does not justify the setup

    • You have existing systems that meet all your needs and you do not plan to change

    • You need features that are not yet available in the open source version

    Implementation Considerations

    If you decide to implement the vCon MCP Server, here are things to consider:

    Database setup - You will need a Supabase account or self-hosted PostgreSQL database. The free tier works for development and small deployments, but production may require a paid tier.

    AI assistant selection - You need an AI assistant that supports MCP. Currently, Claude Desktop supports MCP, with more assistants adding support over time.

    Data migration - If you have existing conversation data, you will need to migrate it to vCon format. The server includes tools and examples for this.

    Training - Teams need to learn how to work with AI assistants and the vCon format. The learning curve is relatively gentle because you interact in natural language, but some training helps.

    Compliance plugins - If you need compliance features, you may need proprietary plugins in addition to the open source server.

    Scaling - For large deployments, consider caching with Redis and potentially running multiple server instances. The architecture supports scaling, but you need to plan for it.

    Integration - Think about how the server fits into your existing systems. The server works well as part of a larger ecosystem, but you need to plan the integrations.

    Getting Started

    If you want to try the vCon MCP Server:

    1. Set up a Supabase account (free tier works for testing)

    2. Install the server following the documentation

    3. Connect Claude Desktop or another MCP-compatible assistant

    4. Start with a small set of test conversations

    The server is open source, so you can evaluate it without commitment. Many organizations start with a pilot project to understand the value before broader deployment.

    Conclusion

    Conversation data is valuable, but realizing that value requires solving problems of fragmentation, vendor lock-in, and limited access. MCP servers provide a standard way for AI assistants to access conversation data, and the vCon format ensures data portability and completeness.

    Organizations use this combination to improve customer service, sales effectiveness, compliance, research, and healthcare documentation. The value comes from better access to insights, automated compliance, reduced integration costs, and future flexibility.

    If you want to learn more about implementing the vCon MCP Server, the earlier posts in this series cover the overview, MCP and AI integration, server capabilities, and architecture in detail.

    What the vCon MCP Server Can Do

    The vCon MCP Server provides over 27 tools that let AI assistants work with conversation data. This post gives you a complete overview of what the server can do, organized by category.

    Core Operations: Managing Conversations

    The server can create, read, update, and delete conversation records. These are the basic operations you need for managing any conversation data.

    Creating Conversations

    You can create new conversation records in several ways:

    Create from scratch - Provide all the conversation data in vCon format, including participants, dialog content, analysis results, and attachments. The server validates everything to ensure it follows the IETF vCon standard before storing it.

    Create from templates - Use predefined templates for common conversation types. The server includes templates for phone calls, chat conversations, email threads, and video meetings. Templates set up the basic structure, and you add the specific details.

    When you create a conversation, the server automatically generates a unique identifier if you do not provide one. It also records when the conversation was created and can track when it was last updated.

    Reading Conversations

    You can retrieve conversations by their unique identifier. The server returns the complete conversation record, including all participants, dialog content, analysis results, and attachments.

    You can also request only specific parts of a conversation. For example, you might want just the participant information, or just the analysis results. This is useful when you do not need the entire conversation and want to reduce the amount of data transferred.

    The server also provides resources, which are URI-based ways to access conversation data. This lets you access specific parts of conversations directly without going through the full tool interface.

    Updating Conversations

    You can update the metadata of conversations, such as the subject line or custom extensions. The server supports different update strategies. You can replace existing values, merge new values with existing ones, or append to existing arrays.

    Updates are validated to ensure the conversation still follows the vCon standard after changes. This prevents accidental corruption of your data.

    Deleting Conversations

    You can delete conversations and all their related data. The server requires explicit confirmation to prevent accidental deletions. When you delete a conversation, it removes all associated participants, dialog entries, analysis results, and attachments.

    Component Management: Adding Parts to Conversations

    A conversation record can have several types of components. You can add or update these components without recreating the entire conversation.

    Dialog Entries

    Dialog entries are the actual conversation content. This might be a phone call recording, a text chat, an email message, or a video call segment. Each dialog entry includes information about when it occurred, who participated, and the content itself.

    The server supports different types of dialog:

    • Recordings of audio or video

    • Text-based conversations

    • Transfers where a call was moved between agents

    • Incomplete conversations where the recording was cut off

    You can add multiple dialog entries to a single conversation, which is useful for conversations that happened across multiple sessions or channels.

    Analysis Results

    Analysis results are insights generated by AI or other processing tools. Examples include transcripts of audio recordings, sentiment analysis, summaries, topic extraction, and named entity recognition.

    Each analysis entry includes information about what tool created it, what version or model was used, and when it was created. This makes it possible to track how analysis results were generated and compare results from different tools.

    The server can store analysis results in different formats. Some might be plain text, while others might be structured JSON data. The server handles both types correctly.

    Attachments

    Attachments are files or documents related to the conversation. This might be an invoice discussed during a call, a screenshot shared during a chat, or a contract sent via email.

    Attachments can be stored in different ways. They can be embedded directly in the conversation record, or they can be referenced by URL if stored elsewhere. The server tracks what type of file each attachment is and can link attachments to specific dialog entries or participants.

    Tags are a special type of attachment used for organization, which we will cover in more detail later.

    Participant Management

    Participants, called parties in the vCon standard, are the people or entities involved in a conversation. Each party has information like name, email address, phone number, role, and organization.

    The server lets you add, update, and manage participants. You can link dialog entries and attachments to specific participants, making it easy to see what each person said or contributed.

    Search Capabilities: Finding What You Need

    The server provides four different ways to search conversations. Each method works better for different types of queries.

    Basic Filter Search

    Basic filter search lets you find conversations by metadata like subject line, participant names, email addresses, phone numbers, or date ranges. This is the fastest search method and works well when you know specific details about what you are looking for.

    For example, you could search for all conversations from last week where a specific customer participated. The search returns complete conversation records that match your criteria.

    Keyword Search

    Keyword search looks for specific words or phrases within the conversation content. It searches through subject lines, dialog text, analysis results, and participant information. This is useful when you remember a specific word or phrase but do not remember other details about the conversation.

    The search uses full-text indexing, which makes it fast even with large numbers of conversations. It also handles minor typos and variations in spelling. Results show snippets of where the keywords were found, making it easy to see the relevant parts.

    You can combine keyword search with filters like date ranges or tags to narrow down results.

    Semantic Search

    Semantic search finds conversations by meaning rather than exact words. It uses AI embeddings to understand what conversations are about and can find related content even when the exact words are different.

    For example, if you search for "customer frustration," semantic search might find conversations where customers were "upset," "angry," "disappointed," or "unhappy," even if those exact words were not used. This makes semantic search useful when you are looking for concepts rather than specific phrases.

    Semantic search requires that conversations have been processed to generate embeddings. The server can generate these embeddings when conversations are created or updated, or you can process existing conversations in batch.

    Hybrid Search

    Hybrid search combines keyword search and semantic search. It uses both methods and combines the results, giving you the benefits of both approaches. Keyword search finds exact matches, while semantic search finds conceptually related content.

    You can control how much weight each method has in the results. For example, you might want exact keyword matches to rank higher than semantic matches, or vice versa.

    Tag System: Organizing Conversations

    Tags are key-value pairs that you can attach to conversations for organization and filtering. Tags are flexible. You can use them however makes sense for your workflow.

    Adding and Managing Tags

    You can add tags individually or in groups. Tags can have string, number, or boolean values. For example, you might tag conversations with department names, priority levels, resolution status, or customer IDs.

    The server provides tools to add, update, get, and remove tags. You can also bulk update tags across multiple conversations, which is useful when you need to reorganize or reclassify conversations.

    Searching by Tags

    All search methods support filtering by tags. You can require conversations to have specific tags, or you can search for conversations that have any of several tags. This makes it easy to narrow down results to the conversations you care about.

    For example, you might search for high-priority sales conversations from last month. The tag system makes this type of query simple and fast.

    Tag Analytics

    The server can analyze your tag usage to show you what tags are used most often, what values are most common, and how tags are distributed across your conversations. This helps you understand how conversations are organized and can guide decisions about tag structure.

    Database Analytics: Understanding Your Data

    The server provides comprehensive analytics about your conversation database. These tools help you understand what you have, how it is growing, and how it is structured.

    Overall Analytics

    You can get analytics about database size, growth trends, content distribution, and health metrics. This includes information like total number of conversations, how many have been added over time, what types of content are stored, and how the database is performing.

    Growth Analytics

    The server can analyze growth patterns over time, showing monthly trends and projections. This helps you understand if your conversation data is growing steadily, if there are seasonal patterns, or if growth is accelerating.

    Content Analytics

    Content analytics show what types of conversations you have, what types of dialog entries are stored, what analysis has been performed, and how participants are distributed. This helps you understand the composition of your conversation data.

    Attachment Analytics

    Attachment analytics show what types of files are stored, how much storage they use, and how they are distributed across conversations. This helps you understand storage usage and plan for capacity.

    Health Metrics

    Health metrics show database performance indicators like query speed, index usage, and cache hit rates. This helps you identify performance issues and optimize your database configuration.

    Database Tools: Inspection and Optimization

    The server includes tools for inspecting and optimizing your database. These are useful for understanding your database structure and ensuring it performs well.

    Database Shape

    You can inspect the database structure to see what tables exist, what indexes are defined, how large tables are, and how tables relate to each other. This is useful for understanding the database schema and debugging issues.

    Database Statistics

    You can get statistics about database usage, including how often tables are accessed, how indexes are being used, and cache performance. This helps identify optimization opportunities.

    Smart Limits

    For large databases, the server can recommend search limits based on database size. This prevents queries from using too much memory or taking too long. The server analyzes your database and suggests appropriate limits for different types of searches.

    Templates and Schemas: Working with Standards

    The server includes support for templates and schemas that make it easier to work with the vCon standard.

    Templates

    Templates provide pre-configured structures for common conversation types. Instead of building everything from scratch, you can start with a template and fill in the specifics. This reduces errors and speeds up creation of new conversation records.

    The server includes templates for phone calls, chat conversations, email threads, video meetings, and custom scenarios.

    Schemas

    You can request the vCon schema in different formats, including JSON Schema and TypeScript types. This is useful when building integrations or writing code that works with vCon data. The schema shows exactly what fields are required, what fields are optional, and what types of values are allowed.

    Examples

    The server can provide example vCon records in different formats. Examples include minimal records, phone calls, chat conversations, emails, video meetings, and full-featured records. These examples serve as references when creating your own conversations.

    Plugin Architecture: Extending the Server

    The server supports plugins that can add custom functionality. Plugins can:

    • Add new tools that the assistant can use

    • Add new resources for data access

    • Intercept operations to add custom logic

    • Implement privacy and compliance features

    Plugins use hooks that fire at different points in the request lifecycle. For example, a plugin might run code before a conversation is created, after it is created, before it is read, or after it is updated. This lets plugins add logging, access control, data transformation, or other functionality.

    The core server focuses on standard conversation management. Plugins extend it for specific needs like compliance, privacy, or integrations.

    What is Included vs What is Not

    The server is open source and includes all the features described above. However, some advanced features are available only through proprietary plugins.

    Included in Open Source

    The open source version includes:

    • Complete CRUD operations for conversations

    • All four search modes

    • Tag management

    • Component management

    Available Through Plugins

    Advanced features available through plugins include:

    • Consent management and tracking

    • Privacy request handling for regulations like GDPR and CCPA

    • Automatic compliance checking

    • Advanced audit logging

    The open source version is fully functional for general conversation data management. The proprietary plugins add features needed for regulated industries or strict compliance requirements.

    Practical Examples

    Here are some practical examples of how you might use these features:

    Example 1: Customer Support Team A support team uses the server to store all customer calls. They tag conversations with issue types, customer IDs, and resolution status. They use semantic search to find similar past issues when helping customers. They add analysis results that include transcripts and sentiment scores.

    Example 2: Sales Team A sales team records all sales calls and meetings. They tag conversations with product names, deal stages, and customer segments. They use keyword search to find conversations where specific products were discussed. They generate summaries automatically and attach them as analysis results.

    Example 3: Compliance Team A compliance team stores all regulated communications. They use tags to track consent status and retention periods. They use database analytics to monitor data growth and ensure compliance with retention policies. They use plugins to add automatic compliance checking and audit logging.

    Example 4: Research Team A research team collects conversation datasets for analysis. They use the server to store conversations in a standard format that works across different tools. They use semantic search to find conversations by research themes. They export data for use in machine learning pipelines.

    Integration Points

    The server integrates with several systems:

    Supabase - The database backend. Supabase provides PostgreSQL with additional features like real-time subscriptions and REST APIs.

    Redis - Optional caching layer. Redis can be added to dramatically speed up reads of frequently accessed conversations.

    OpenAI - Optional for semantic search. The server can use OpenAI's API to generate embeddings for semantic search, though local models can also be used.

    AI Assistants - Any assistant that supports MCP can connect to the server. Currently this includes Claude Desktop, with more assistants adding support over time.

    Conclusion

    The vCon MCP Server provides a comprehensive set of tools for managing conversation data. It covers the basics of creating and managing conversations, advanced search capabilities, organization through tags, analytics for understanding your data, and extensibility through plugins.

    The next post in this series covers the architecture of the server, explaining how it is built and why it is designed the way it is.

    Standard Links

    Available Links

    Name
    Description

    analyze

    A powerful OpenAI-based analysis link that performs AI-powered analysis on vCon transcripts. Supports customizable analysis types, prompts, and models with configurable sampling and retry mechanisms.

    analyze_vcon

    A specialized link that performs AI-powered analysis on entire vCon objects, returning structured JSON output. Unlike the standard analyze link, it processes the complete vCon structure rather than individual dialogs, with support for system prompts and JSON response validation.

    Concepts

    Important Ideas for vCons and the Conserver

    vCon

    A vCon is the container for data and information relating to a real-time, human conversation. It is analogous to a [] which enables the definition, interchange and storage of an individual's various points of contact. The data contained in a vCon may be derived from any multimedia session, traditional phone call, video conference, SMS or MMS message exchange, webchat or email thread. The data in the container relating to the conversation may include Call Detail Records (CDR), call meta data, participant identity information (e.g. STIR PASSporT), the actual conversational data exchanged (e.g. audio, video, text), realtime or post conversational analysis and attachments of files exchanged during the conversation. A standardized conversation container enables many applications, establishes a common method of storage and interchange, and supports identity, privacy and security efforts

    Operational Benefits of Conservers

    Enabling Flexible, Secure, and Scalable Conversation Intelligence Infrastructure

    Introduction

    In an era where conversational data represents one of the most valuable yet challenging assets for enterprises, the Conserver platform emerges as a critical infrastructure component that bridges the gap between raw communication data and actionable intelligence. Built upon the open vCon (Virtual Conversation) standard, Conservers provide organizations with unprecedented operational flexibility in how they capture, process, analyze, and manage conversation data while maintaining strict security and compliance requirements.

    The operational benefits of Conservers extend far beyond simple conversation recording or transcription. They represent a fundamental shift in how organizations can architect their conversation intelligence infrastructure, offering the flexibility to adapt to diverse deployment requirements, security constraints, and processing needs while maintaining a consistent, standardized approach to conversation data management.

    MCP and Conserver Together

    Different Roles, Working Together

    The vCon ecosystem includes two main components: the MCP server and the conserver. They serve different purposes but work together to create a complete system for managing conversation data. This post explains what each does, how they differ, and how they combine with Redis and Supabase to optimize performance.

    Two Different Roles

    The MCP server and conserver solve different problems in the vCon ecosystem.

    The MCP server is a storage and query system. It provides an interface for AI assistants to work with conversation data. It focuses on storing, searching, and retrieving vCons through the Model Context Protocol.

    How the vCon MCP Server is Built

    The vCon MCP Server is built in layers, with each layer handling a specific responsibility. This architecture makes the server reliable, performant, and extensible. This post explains how it works.

    The Three-Layer Architecture

    The server has three main layers:

    MCP Server Layer - Handles communication with AI assistants using the Model Context Protocol. This layer exposes tools, resources, and prompts that assistants can use.

    Business Logic Layer - Contains the core functionality for managing conversations. This includes query engines, validation, and plugin systems.

    Database Layer

    You want to integrate conversation data with other business systems

  • You are building applications that work with conversation data

  • Experiment with asking the assistant to work with your data
  • Gradually expand as you see value

  • Add integrations with other systems
    Database analytics
  • Templates and schemas

  • Basic plugin architecture

  • PII detection and masking
  • Data retention enforcement

  • Regulatory compliance workflows

  • Deployment Flexibility: From Cloud to Air-Gapped Environments

    One of the most significant operational benefits of Conservers is their ability to support virtually any deployment model an organization requires. Conservers can operate in:

    • Cloud-Native Deployments: Leveraging AWS, Azure, or Google Cloud Platform services for maximum scalability and managed services integration

    • On-Premises Installations: Complete local deployment with air-gapped capabilities for sensitive environments requiring total data isolation

    • Hybrid Configurations: Flexible combination of cloud and on-premises components to balance performance, cost, and security requirements

    • Edge Deployments: Distributed processing at edge locations with local GPU utilization for real-time processing without backhauling data

    This flexibility is particularly crucial for organizations operating under strict data sovereignty requirements or in highly regulated industries. For instance, a healthcare organization might deploy Conservers entirely behind their firewall to process patient conversations, ensuring HIPAA compliance while still leveraging advanced AI capabilities through local GPU processing. Government agencies can maintain classified conversation processing within secure facilities while using less sensitive Conservers in standard environments.

    Seamless AI Model Selection and Integration

    Perhaps one of the most powerful operational benefits is the Conserver's ability to seamlessly switch between different AI processing options based on specific requirements. Organizations can choose between:

    Cloud-Based AI Services

    Integration with industry-leading services such as OpenAI GPT-4 for advanced language understanding, Deepgram Nova-2 for state-of-the-art transcription accuracy, or Groq Whisper for high-speed audio processing. These services provide cutting-edge capabilities without the need for local infrastructure investment.

    On-Premises AI Processing

    Complete data sovereignty through local deployment using open-source models from Hugging Face, custom-trained proprietary models, or specialized AI implementations. This approach ensures that sensitive conversation data never leaves the organization's security perimeter while still enabling advanced AI analysis.

    Hybrid AI Processing

    A sophisticated approach where sensitive data elements are processed locally while non-sensitive analysis leverages cloud services. For example, personally identifiable information (PII) can be detected and redacted on-premises before sending sanitized data to cloud services for broader analysis.

    This flexibility enables organizations to optimize for performance, cost, compliance, or security on a per-conversation or per-processing-step basis. Financial institutions might process trading floor conversations entirely on-premises for regulatory compliance while using cloud services for customer service interactions. Healthcare providers can ensure patient privacy by keeping all medical conversations within their infrastructure while leveraging cloud AI for administrative communications.

    Chain-Based Processing Architecture

    The modular, chain-based processing architecture of Conservers provides exceptional operational flexibility. Organizations can configure processing chains that match their exact requirements, creating sophisticated workflows that adapt to different conversation types, compliance needs, or business objectives.

    This architecture enables organizations to:

    • Create Specialized Processing Pipelines: Design unique processing flows for different conversation types, such as sales calls versus support interactions

    • Implement A/B Testing: Compare different processing approaches to optimize for accuracy, cost, or speed

    • Scale Individual Components: Independently scale specific processing steps based on bottlenecks or demand

    • Add or Remove Processing Steps: Modify workflows without system redesign or downtime

    For example, a contact center might implement different processing chains for:

    • Sales Conversations: Emphasis on opportunity identification and sentiment analysis

    • Support Calls: Focus on issue resolution tracking and customer satisfaction

    • Compliance Recordings: Priority on regulatory requirement verification and audit trail creation

    Multi-Conserver Orchestration

    One of the most innovative operational benefits is the ability to chain multiple Conservers together, creating sophisticated processing topologies that respect security boundaries while maximizing processing capabilities. This pattern enables several powerful operational scenarios:

    Security Boundary Traversal

    Organizations can deploy Conservers at different security levels, with each maintaining exclusive access to AI assets within its boundary. A typical deployment might include:

    • DMZ Conserver: Handles initial conversation ingestion and basic processing with limited AI capabilities

    • Internal Network Conserver: Performs enhanced analysis using corporate AI models and proprietary algorithms

    • Classified Environment Conserver: Executes specialized processing using sensitive AI models and classified analysis techniques

    Each Conserver processes the conversation to the extent allowed by its security context, then passes the enhanced vCon to the next level, building a comprehensive analysis while maintaining strict security segregation.

    Geographic Distribution

    Multiple Conservers can be deployed across different geographic regions to comply with data residency requirements while maintaining global processing capabilities:

    • European Conserver: Processes EU citizen conversations in compliance with GDPR, keeping data within EU boundaries

    • North American Conserver: Handles US and Canadian conversations with appropriate CCPA and PIPEDA compliance

    • Asia-Pacific Conserver: Manages regional conversations in compliance with local data protection laws

    This distributed approach enables global organizations to maintain a unified conversation intelligence platform while respecting regional regulations and data sovereignty requirements.

    Operational Scalability and Performance

    The Conserver architecture provides multiple mechanisms for operational scaling:

    Horizontal Scaling

    The stateless application design enables organizations to add processing nodes as demand increases. The Redis-based queue management system automatically distributes load across available processors, while Kubernetes orchestration provides auto-scaling based on real-time demand metrics.

    Vertical Scaling

    Individual components can be scaled independently based on their specific resource requirements. GPU resources can be allocated dynamically for AI processing tasks, while storage backends can be optimized for specific workload patterns such as high-write throughput or complex query operations.

    Performance Optimization

    Organizations can implement sophisticated performance optimization strategies:

    • Tiered Caching: Hot data in Redis for microsecond access, warm data in PostgreSQL for structured queries, and cold data in S3 for long-term archival

    • Batch Processing: Aggregate multiple conversations for efficient processing during off-peak hours

    • Intelligent Routing: Direct conversations to appropriate processing resources based on priority, type, or urgency

    Multi-Tenant Operations

    Conservers provide sophisticated multi-tenant capabilities that enable service providers and enterprises to operate efficiently at scale. Each tenant receives:

    • Complete Data Isolation: Ensuring that one tenant's conversations never mix with another's

    • Customized Compliance Profiles: Tailored to specific industry requirements or geographic regulations

    • Independent Processing Pipelines: Allowing each tenant to configure their own processing workflows

    • Dedicated Resource Allocation: Guaranteeing performance levels through resource reservation

    This multi-tenant architecture enables Unified Communications as a Service (UCaaS) providers to offer conversation intelligence capabilities to their entire customer base while maintaining strict separation and customization for each client.

    Compliance and Audit Operations

    The operational benefits extend significantly into compliance and audit operations:

    Automated Compliance Workflows

    Conservers automatically enforce compliance policies based on conversation metadata, participant information, or content analysis. For GDPR compliance, the system can automatically enforce data retention periods, process "Right to Be Forgotten" requests, and maintain comprehensive consent records. For HIPAA compliance, it ensures encryption standards, access logging, and automated PHI detection with appropriate handling.

    Immutable Audit Trails

    Using SCITT (Supply Chain Integrity, Transparency and Trust) protocol, Conservers create cryptographically verified audit trails that provide:

    • Tamper-Proof Activity Records: Every action is recorded with cryptographic proof of integrity

    • Regulatory Compliance Evidence: Ready-to-submit reports for regulatory audits

    • Transparent AI Decision Documentation: Complete lineage of AI processing and decision-making

    • Long-Term Compliance Verification: Archived audit trails that remain verifiable years later

    Operational Cost Optimization

    Conservers enable sophisticated cost optimization strategies through their flexible architecture:

    Tiered Processing Strategies

    Organizations can implement cost-effective processing by:

    • Sampling: Process a representative sample of conversations for quality assurance rather than 100% coverage

    • Priority-Based Processing: Apply expensive AI analysis only to high-value conversations

    • Off-Peak Scheduling: Batch non-urgent processing during lower-cost computing periods

    • Progressive Enhancement: Start with basic processing and add advanced analysis only when specific triggers are detected

    Storage Optimization

    Multi-tier storage strategies significantly reduce costs while maintaining performance:

    • Hot Storage: Recent conversations in high-speed Redis cache for immediate access

    • Warm Storage: Active conversations in PostgreSQL for complex queries and reporting

    • Cold Storage: Archived conversations in S3 Glacier for compliance and long-term retention

    Operational Resilience

    The Conserver architecture provides multiple layers of operational resilience:

    Failure Recovery Mechanisms

    • Automatic Retry Logic: Failed processing attempts are automatically retried with exponential backoff

    • Dead Letter Queue Management: Conversations that repeatedly fail processing are captured for manual review

    • Fallback Processing Paths: Alternative processing routes activate when primary paths fail

    • Multi-Storage Redundancy: Data is replicated across multiple storage backends for durability

    Disaster Recovery Capabilities

    Organizations can implement comprehensive disaster recovery strategies with:

    • Geographic Replication: Conversations replicated across multiple regions

    • Automated Failover: Seamless switching to backup sites during outages

    • Point-in-Time Recovery: Ability to restore to any previous state

    • Regular DR Testing: Automated testing of recovery procedures without impacting production

    Real-World Operational Scenarios

    Financial Services Deployment

    A global investment bank leverages Conservers to create a comprehensive compliance and intelligence platform:

    • Trading floor communications are processed entirely on-premises using local GPU clusters for real-time compliance monitoring

    • Customer service conversations utilize cloud AI for sentiment analysis and satisfaction tracking

    • All conversations maintain immutable audit trails for regulatory review spanning seven years

    • Real-time compliance alerts trigger when potential violations are detected

    Healthcare System Implementation

    A multi-hospital healthcare network uses Conservers to transform patient care:

    • Patient conversations are processed within HIPAA-compliant on-premises infrastructure

    • Local GPU resources power medical transcription with specialized vocabulary

    • Department-specific Conservers handle specialized processing for radiology, pathology, and emergency departments

    • Automated clinical documentation reduces physician administrative burden by 60%

    Global Contact Center

    A multinational contact center with 50,000 agents deploys Conservers to enhance customer experience:

    • Regional Conservers process conversations in local data centers for compliance

    • Local language models provide accurate transcription in 30+ languages

    • Global insights are aggregated while respecting data residency requirements

    • Dynamic scaling handles seasonal peaks with 10x normal volume

    Future-Proofing Operations

    Conservers provide several mechanisms for future-proofing operational investments:

    Technology Evolution

    The modular architecture ensures that new AI models, storage technologies, or processing techniques can be integrated without redesigning the system. As new language models emerge or transcription technologies improve, they can be added as new link processors while maintaining existing workflows.

    Regulatory Adaptation

    The flexible compliance framework allows organizations to adapt to new regulations without major system changes. New consent requirements, data handling rules, or audit requirements can be implemented through configuration updates rather than code changes.

    Scale Preparation

    The architecture supports growth from thousands to millions of conversations without fundamental changes. Organizations can start small and scale up by adding resources rather than replacing systems.

    Conclusion

    The operational benefits of Conservers represent a paradigm shift in how organizations approach conversation intelligence infrastructure. By providing unprecedented flexibility in deployment models, AI processing options, and architectural patterns, Conservers enable organizations to build sophisticated conversation processing systems that adapt to their specific operational requirements rather than forcing them into rigid, one-size-fits-all solutions.

    The ability to seamlessly choose between cloud and on-premises AI processing, chain multiple Conservers across security boundaries, and maintain complete operational control while leveraging best-in-class capabilities makes Conservers an essential component of modern enterprise infrastructure. As organizations continue to recognize the value locked within their conversational data, the operational flexibility and sophistication provided by Conservers will become increasingly critical to maintaining competitive advantage while ensuring security, compliance, and operational excellence.

    Through their modular architecture, open standards foundation, and comprehensive operational capabilities, Conservers demonstrate that organizations need not choose between powerful AI capabilities and operational control. Instead, they can have both, configured and deployed in ways that match their unique operational requirements, security constraints, and business objectives. This operational flexibility, combined with enterprise-grade reliability and security, positions Conservers as the foundational infrastructure for the next generation of conversation intelligence applications.

    datatrails

    A specialized link that integrates vCon data with the DataTrails platform, creating verifiable audit trails for vCon operations. Supports both asset-based and asset-free events, with automatic token management and structured event attributes mapping to SCITT envelopes.

    deepgram_link

    A specialized link that performs speech-to-text transcription on audio recordings in vCon dialogs using the Deepgram API. Supports automatic language detection, confidence scoring, and minimum duration filtering for transcription quality.

    diet

    A specialized link that helps reduce the size and content of vCon objects by selectively removing or modifying specific elements. Useful for data minimization, privacy protection, and optimizing storage with options for media redirection and system prompt removal.

    expire_vcon

    A simple but effective link that sets an expiration time for vCon objects in Redis, enabling automatic cleanup after a specified period. Useful for data retention policy enforcement and storage optimization.

    groq_whisper

    A specialized link that performs speech-to-text transcription on audio recordings in vCon dialogs using Groq's implementation of the Whisper ASR service, with support for various audio formats and robust error handling.

    hugging_face_whisper

    A specialized link that performs speech-to-text transcription on audio recordings in vCon dialogs using Hugging Face's implementation of the Whisper ASR service, with support for various audio formats and robust error handling.

    hugging_llm_link

    A specialized link that performs AI-powered analysis on vCon transcripts using Hugging Face's language models, supporting both API-based and local model inference with configurable parameters and robust error handling.

    jq_link

    A specialized link that filters vCon objects using jq expressions, allowing for complex content-based filtering with configurable forwarding rules for matching or non-matching vCons.

    post_analysis_to_slack

    A specialized link that posts vCon analysis results to Slack channels, with support for team-specific notifications, conditional posting based on analysis content, and rich message formatting.

    sampler

    A specialized link that selectively processes vCons based on various sampling methods, including percentage, rate, modulo, and time-based sampling, enabling efficient resource utilization and focused analysis.

    scitt

    A specialized link that provides integrity and inclusion protection for vCon objects by creating and registering signed statements on a SCITT Transparency Service, ensuring vCons can be verified as authentic and complete.

    tag

    A simple link that adds configurable tags to vCon objects, enabling better organization and filtering of conversations.

    tag_router

    A specialized link that routes vCon objects to different Redis lists based on their tags, enabling flexible distribution of conversations to different processing queues or storage locations.

    webhook

    A specialized link that sends vCon objects to configured webhook URLs, enabling integration with external systems and event-driven workflows.

    Conserver

    The conserver is a data platform designed to extract conversations from business phone systems, transform them into actionable insights, and send that data into common business tools such as spreadsheets, Salesforce and no code toolsets. An open core product, the conserver enables data engineering teams to supply a reliable source of information for AI, ML and operational software in cloud, premise and hybrid contexts. The core for many of the business cases enabled by the conserver is the smart capture, redaction and lifecycle management of recorded customer conversations and customer journeys, recently accelerated by FTC and GDPR regulations and by increasing investments into AI and ML.

    From a system perspective, shown above, the Conserver attaches to information systems like Web Chat and call center queues, and extracts information from them after conversations are ended. This information is then cleaned and transformed into actionable data. For instance, a distributed call center might extract conversations from a group of sales agents, convert them into text, then filter those conversations looking for times when customers objected to a sale. These objections are then pushed into database tables and Google Sheets as a data self-service option for any business team. The conserver supports multiple data pipelines, each one extracting data from a number of systems, performing transformations such as translations, transcriptions and redactions, and then pushing the prepared data into applications to be used.

    In contrast to other data platforms, the Conserver is dedicated to managing the particular complexities of real time conversational sources. For instance, the amount of bandwidth and storage required to manage an hour long audio recording is an order of magnitude larger than managing a typical business object like a PDF. However, even this is just a start. Video is a few orders of magnitude greater than that, and the data creation for service providers such as Zoom and Skype are magnitudes of order still greater. From a legal perspective, regulatory compliance for customer data protections are particular for recorded conversations, and require support for tracking data’s use by automations, and for tracking deletion from a “Right to be Forgotten” request.

    Parties

    The parties section of the vCon is an array that refers to the people or systems in the conversation. Each "dialog" provides an index into the parties array to identify the people. Each party identifies the

    • Network identifier of the party, currently held in the 'tel" field

    • The mail adress

    • The name of the party

    • The role of the party, represented by a string (we use labels like customer, agent)

    • A validation field, allowing for evidence of third party validation of the identities of the parties

    • Other information, such as civic address, timezone or jCard.

    Dialogs

    The dialogs section of the vCon is an array of transcripts and recordings that represent the media of the conversation itself. Each dialog contains an identification of:

    • The type of the dialog (recording, transcript)

    • The start time

    • Duration

    • The parties in the conversation

    • The originating party of the conversation

    • The mimetype of the recording

    • Any associated filename

    The content of the dialog comes in two flavors: packed and unpacked. Packed data is included in the vCon itself in the body field. Unpacked data is not included in the vCon, but is instead referenced by URL, not necessarily publicly accessible. For both cases, the media of the vCon can be signed to prevent tampering or modification after the vCon is constructed.

    Analysis

    The analysis section of the vCon is an array of objects that represents third party analysis of the vCon itself. Examples of Analysis includes:

    • Sentiment analysis of the conversation itself

    • A list of promises made by the people on the call

    • A summary of the conversation

    Each analysis captures the vendor, schema and details of the analysis itself. In addition to the value that the analysis provides, this also becomes an accounting of the times and places this conversation has been processed by third parties. This list is critical in compliance to data regulations as it allows data controllers to fulfill their obligations to reporting and removing personal data on demand of people and regulators.

    Attachments

    The attachments section of the vCon is an array of objects that are documents, traces and other pieces of data that provides the context of a conversation. For instance, a sales organization may store the lead information in the attachment; a conference call may include the powerpoint that was discussed. Links in the conserver use attachments to store tracing information, such as the raw responses from external systems or the original source of the vCon.

    Data Privacy

    Data privacy, also known as information privacy or data protection, refers to the practice of safeguarding individuals' personal information from unauthorized access, use, disclosure, alteration, or destruction. It involves ensuring that individuals have control over their own personal data and that organizations that collect, store, and process personal data do so in a manner that respects individuals' privacy rights.

    Key concepts and principles of data privacy include:

    1. Consent: Organizations should obtain individuals' informed consent before collecting, using, or sharing their personal data. Consent should be freely given, specific, informed, and unambiguous.

    2. Purpose Limitation: Personal data should be collected for specific, explicit, and legitimate purposes, and it should not be used for purposes that are incompatible with those for which it was originally collected.

    3. Data Minimization: Organizations should collect only the minimum amount of personal data necessary to achieve the stated purpose. Excessive or irrelevant data should not be collected.

    4. Accuracy: Personal data should be accurate, complete, and up-to-date. Organizations should take reasonable steps to correct or delete inaccurate data.

    5. Storage Limitation: Personal data should be retained only for as long as necessary to fulfill the purposes for which it was collected. Organizations should establish retention policies and securely dispose of data that is no longer needed.

    6. Security: Organizations should implement appropriate technical and organizational measures to protect personal data from unauthorized access, data breaches, theft, and other security risks. This may include encryption, access controls, and regular security assessments.

    7. Transparency: Organizations should be transparent about their data collection and processing practices. This includes providing clear privacy policies and notices that inform individuals about how their data is being used and their rights regarding their data.

    8. Individual Rights: Individuals have certain rights regarding their personal data, including the right to access their data, the right to request corrections or deletions, the right to object to certain uses of their data, and the right to data portability (the ability to transfer their data from one organization to another).

    9. Accountability: Organizations are responsible for complying with data privacy laws and regulations and for demonstrating their compliance. This may involve conducting privacy impact assessments, appointing a data protection officer, and maintaining records of data processing activities.

    Data privacy laws and regulations vary by jurisdiction, but many countries and regions have enacted legislation to protect individuals' personal data. Examples of data privacy laws include the General Data Protection Regulation (GDPR) in the European Union, the California Consumer Privacy Act (CCPA) in the United States, and the Personal Data Protection Act (PDPA) in Singapore.

    Overall, data privacy is a critical aspect of modern society, as the collection and use of personal data have become pervasive in many aspects of life, including online services, healthcare, finance, and commerce. Respecting data privacy helps build trust between individuals and organizations and protects individuals' rights and freedoms.

    Data Minimization

    Data minimization is a privacy and data protection principle that aims to limit the collection, processing, and retention of personal data to what is strictly necessary for a specific purpose. The principle of data minimization is based on the idea that organizations should only collect, use, and store the minimum amount of personal data required to achieve a legitimate purpose, and should avoid collecting or retaining excessive or irrelevant data.

    Data minimization is a key component of many privacy laws and regulations, including the European Union's General Data Protection Regulation (GDPR). It is closely related to other data protection principles, such as purpose limitation and storage limitation.

    The principle of data minimization can be broken down into three main components:

    1. Collection Minimization: Organizations should only collect personal data that is directly relevant and necessary for the intended purpose. They should avoid collecting data that is not needed for the specific purpose for which it is being collected. For example, if a company is collecting data for the purpose of processing an online order, it should not collect information about the customer's hobbies or political preferences, as this information is not relevant to the transaction.

    2. Use Minimization: Organizations should only use personal data for the specific purpose for which it was collected and should avoid using the data for other purposes unless there is a legitimate basis for doing so. For example, if a company collects a customer's email address for the purpose of sending a receipt, it should not use that email address to send marketing materials unless the customer has explicitly consented to receive such communications.

    3. Retention Minimization: Organizations should only retain personal data for as long as necessary to fulfill the specific purpose for which it was collected. Once the data is no longer needed for that purpose, it should be securely deleted or anonymized. For example, if a company collects credit card information to process a one-time payment, it should not retain that information indefinitely after the payment has been processed.

    Implementing data minimization practices can help organizations reduce the risk of data breaches, protect individuals' privacy rights, and comply with legal and regulatory requirements. Additionally, data minimization can lead to cost savings by reducing the amount of data that needs to be stored and managed, and it can help organizations build trust with their customers and stakeholders by demonstrating a commitment to privacy and data protection.

    Data Protection

    Data protection is closely related to data privacy, and the terms are sometimes used interchangeably. However, data protection is more focused on the technical and organizational measures that organizations take to secure personal data, while data privacy encompasses the broader set of legal, ethical, and social considerations related to the collection and use of personal data.

    Consent

    Customer consent refers to the process by which a customer or user gives their explicit and informed permission for an organization to collect, use, or disclose their personal data for specific purposes. Consent is a fundamental concept in privacy and data protection laws, and it is one of the legal bases that organizations can rely on to process personal data.

    Key aspects of customer consent include the following:

    1. Freely Given: Consent must be given voluntarily, without coercion or undue pressure. Customers should have a genuine choice to either give or withhold their consent. If consent is required as a condition for accessing a service, and there is no reasonable alternative, the consent may not be considered freely given.

    2. Specific: Consent must be specific to the particular purpose or purposes for which the data will be used. Organizations should clearly explain the purpose of data collection and processing, and consent should not be sought for broad or undefined purposes.

    3. Informed: Customers must be provided with clear and understandable information about what data will be collected, how it will be used, and with whom it may be shared. This information should be presented in a transparent and easily accessible manner, so that customers can make an informed decision about whether to give their consent.

    4. Unambiguous: Consent must be expressed through a clear, affirmative action. This means that customers must actively indicate their agreement, such as by ticking a box, signing a form, or clicking a button. Passive or implied consent (e.g., pre-ticked boxes or inactivity) is not considered valid consent.

    5. Revocable: Customers have the right to withdraw their consent at any time, and the process for doing so should be straightforward and easily accessible. Organizations must inform customers of their right to withdraw consent and provide a mechanism for doing so.

    6. Documented: Organizations should keep a record of when and how consent was obtained, as well as the specific information that was provided to the customer at the time of consent. This documentation can help demonstrate compliance with legal and regulatory requirements.

    It's important to note that there are certain situations where consent may not be the appropriate legal basis for processing personal data. For example, consent may not be valid if there is a significant power imbalance between the organization and the individual (e.g., in an employer-employee relationship), or if the processing is necessary to fulfill a legal obligation or a contract.

    Additionally, some data protection laws, such as the European Union's General Data Protection Regulation (GDPR), provide enhanced protections for certain categories of sensitive personal data (e.g., health information, religious beliefs, sexual orientation), and additional requirements may apply when seeking consent for the processing of such data.

    Encryption and Signing

    vCons support encryption of the parent object, and each of the analysis sections can carry encrypted bodies. In addition, external URLs and the vCon itself are signed, allowing for tamper detection of the contents.

    Data Projection

    Since the vCon is a nested document, sometimes it is more convenient to format the data in a vCon in other formats, particularly for relational data storage. For instance, a call log link may need to provide a single database row for each vCon into a spreadsheet. A data projection picks and chooses the data inside the vCon to create the spreadsheet row. Normally, data is lost through a projection.

    Sensitive Personal Data

    The General Data Protection Regulation (GDPR) specifically mentions DNA, health records, and biometric data as types of personal data that are considered "special categories of personal data" (often referred to as "sensitive personal data"). These special categories of personal data are subject to additional protections under the GDPR due to their sensitive nature and the potential risks associated with their processing.

    Article 9 of the GDPR addresses the processing of special categories of personal data, which includes the following types of data:

    • Racial or ethnic origin

    • Political opinions

    • Religious or philosophical beliefs

    • Trade union membership

    • Genetic data (which includes DNA)

    • Biometric data (where processed to uniquely identify a person)

    • Health data (which includes health records)

    • Data concerning a person's sex life or sexual orientation

    The processing of special categories of personal data is generally prohibited under the GDPR, with certain exceptions. Article 9(2) of the GDPR provides a list of specific conditions under which the processing of special categories of personal data may be permitted. These conditions include, but are not limited to, the following:

    • The data subject has given explicit consent to the processing for one or more specified purposes (unless prohibited by EU or member state law).

    • The processing is necessary for the purposes of carrying out the obligations and exercising specific rights of the data controller or data subject in the field of employment, social security, and social protection law (subject to certain safeguards).

    • The processing is necessary to protect the vital interests of the data subject or another person where the data subject is physically or legally incapable of giving consent.

    • The processing relates to personal data that has been made public by the data subject.

    • The processing is necessary for the establishment, exercise, or defense of legal claims or for courts acting in their judicial capacity.

    • The processing is necessary for reasons of substantial public interest, based on EU or member state law.

    • The processing is necessary for medical diagnosis, the provision of health or social care, or the management of health or social care systems (subject to certain safeguards).

    • The processing is necessary for reasons of public interest in the area of public health (subject to certain safeguards).

    Organizations that process special categories of personal data must comply with these additional requirements and ensure that they have a valid legal basis for processing such data. They must also implement appropriate safeguards to protect the rights and freedoms of data subjects.

    vCard
    The conserver is a processing and workflow engine. It focuses on creating and enriching vCons through chains of processing steps. It handles the intake of new conversations and runs them through workflows like transcription, analysis, and enrichment.

    Think of it this way: the conserver is like a factory that creates and processes products, while the MCP server is like a warehouse where products are stored and retrieved. They work together but have different jobs.

    The MCP Server: Storage and Query

    The MCP server is built in TypeScript and serves as the storage backend for conversation data. Its primary role is providing AI assistants with tools to work with stored conversations.

    What It Does

    The MCP server provides:

    • Storage for vCon data in Supabase

    • Search capabilities across stored conversations

    • Query tools for AI assistants

    • Analytics about your conversation database

    • Management tools for organizing conversations

    How It Works

    When you use an AI assistant with the MCP server, the assistant can:

    • Store conversations you create

    • Search through historical conversations

    • Analyze patterns in your data

    • Answer questions about conversations

    • Manage tags and organization

    The server exposes over 27 tools through the MCP protocol. AI assistants understand these tools and can use them to work with your conversation data. The server handles validation, database operations, and response formatting.

    Architecture

    The MCP server uses a cache-first read strategy when Redis is configured. When you request a conversation:

    1. It checks Redis cache first

    2. If found in cache, returns immediately

    3. If not found, fetches from Supabase

    4. Caches the result in Redis for future requests

    5. Returns the data

    This makes reads very fast for frequently accessed conversations.

    The Conserver: Scaling Creation and Processing

    The conserver is built in Python and serves as a workflow engine for processing conversations. Its primary role is taking in raw conversation data and running it through processing chains.

    What It Does

    The conserver provides:

    • Processing chains that can transcribe, analyze, and enrich conversations

    • Scalable processing through multiple instances

    • Integration with external services like transcription APIs

    • Storage backends that work with various databases

    • Queue management for processing workflows

    How It Works

    The conserver processes conversations through configurable chains. A chain is a sequence of processing steps called links. For example, a chain might:

    1. Receive a new conversation

    2. Transcribe audio using Deepgram

    3. Analyze sentiment using OpenAI

    4. Extract key topics

    5. Store the enriched conversation

    You configure chains in a YAML file, defining what processing should happen and in what order. The conserver reads from Redis queues, processes conversations through the chains, and writes results to storage backends.

    Scaling

    The conserver scales by running multiple instances. You can start multiple conserver processes, and they all read from the same Redis queues. This allows you to process many conversations in parallel.

    For example, if you have 100 conversations to process and each takes 30 seconds, one conserver instance would take 50 minutes. With 10 instances, it takes 5 minutes. You scale simply by starting more instances.

    How They Work Together

    The MCP server and conserver complement each other:

    Conserver creates and processes - It takes in raw conversation data, runs it through processing chains, and stores the enriched results.

    MCP server stores and queries - It provides the storage backend and gives AI assistants tools to work with the stored data.

    You can use conserver's Supabase storage backend, which stores data in the same Supabase database that the MCP server uses. This means:

    • Conserver writes enriched conversations to Supabase

    • MCP server can immediately query those conversations

    • Both use the same Redis cache for performance

    • Data flows seamlessly between systems

    The Redis and Supabase Combination

    Using Redis and Supabase together optimizes performance throughout the conversation lifecycle. Here is how it works at each stage.

    Stage 1: Creation and Processing

    When conserver processes a conversation:

    1. Conserver receives the conversation from an API or queue

    2. Conserver processes it through chains (transcription, analysis, etc.)

    3. Conserver writes to Supabase first (permanent storage)

    4. Conserver caches in Redis (fast access)

    5. Conserver adds to queues for further processing if needed

    This is called a write-through cache pattern. Data is written to permanent storage first, then cached. This ensures data is never lost, even if Redis fails.

    The conserver can run multiple instances in parallel, all writing to the same Supabase database. Redis coordinates the work by managing queues that distribute processing across instances.

    Stage 2: Storage and Retrieval

    When the MCP server retrieves a conversation:

    1. MCP server checks Redis first (cache-first read)

    2. If found, returns immediately (typically 1-2 milliseconds)

    3. If not found, fetches from Supabase (typically 50-100 milliseconds)

    4. Caches result in Redis for future requests

    5. Returns the data

    This is called a cache-first read pattern. It checks fast storage first, only going to slower storage when needed. This makes repeated reads very fast.

    Stage 3: Updates and Deletes

    When conversations are updated or deleted:

    1. Operation happens in Supabase (permanent storage)

    2. Cache is invalidated in Redis (removed or marked stale)

    3. Next read fetches fresh data from Supabase and caches it

    This ensures cached data stays consistent with permanent storage.

    Performance Benefits Throughout the Lifecycle

    The Redis and Supabase combination provides speed optimizations at every stage:

    During creation - Conserver writes to Supabase, then caches in Redis. This makes the conversation immediately available for fast reads while ensuring it is permanently stored.

    During processing - Redis queues coordinate work across multiple conserver instances. This allows parallel processing without conflicts.

    During reads - Cache-first reads make frequently accessed conversations load 20 to 50 times faster. A conversation read from cache takes 1-2 milliseconds instead of 50-100 milliseconds.

    During searches - While search always queries Supabase for complete results, cached individual conversations load instantly when displayed.

    During updates - Updates write to Supabase first, then invalidate cache. The next read gets fresh data and caches it, keeping everything consistent.

    A Complete Example

    Let us trace a conversation through the entire system:

    Hour 1: Creation

    • A phone call comes in to your system

    • Conserver receives the call recording

    • Conserver processes it: transcribes audio, analyzes sentiment, extracts topics

    • Conserver writes the enriched vCon to Supabase

    • Conserver caches it in Redis with a 1-hour TTL

    • The conversation is now stored and available

    Hour 2: First Access

    • You ask your AI assistant to find the conversation

    • MCP server checks Redis cache (miss, it expired)

    • MCP server fetches from Supabase (50ms)

    • MCP server caches in Redis (1ms)

    • MCP server returns to assistant (51ms total)

    Hour 3: Repeated Access

    • You ask about the same conversation again

    • MCP server checks Redis cache (hit)

    • MCP server returns immediately (1ms total)

    Hour 4: Update

    • You add analysis results to the conversation

    • MCP server updates Supabase

    • MCP server invalidates Redis cache

    • The conversation is now updated permanently

    Hour 5: Search

    • You search for conversations about a topic

    • MCP server searches Supabase (query always hits database)

    • Results include the conversation you updated

    • When you open individual results, they load from cache if recently accessed

    Configuration Overview

    To use both systems together, you configure:

    Conserver configuration (in config.yml):

    MCP server configuration (in .env):

    Both use the same Supabase database and Redis cache, so data flows seamlessly between them.

    When to Use Each

    Use the conserver when:

    • You need to process many conversations at scale

    • You want to run workflows like transcription and analysis

    • You need to integrate with external processing services

    • You are receiving conversations from APIs or queues

    Use the MCP server when:

    • You want AI assistants to work with conversation data

    • You need to search and query stored conversations

    • You want analytics about your conversation database

    • You are building applications that interact with conversations

    Most organizations use both:

    • Conserver handles the creation and processing pipeline

    • MCP server handles storage and AI assistant access

    • They share the same Supabase database and Redis cache

    Summary

    The MCP server and conserver serve different but complementary roles. The conserver scales the creation and processing of conversations through parallel instances and workflow chains. The MCP server provides storage and query capabilities for AI assistants.

    Together with Redis and Supabase, they create an optimized system:

    • Conserver writes to Supabase first, then caches in Redis (write-through)

    • MCP server reads from Redis first, then Supabase if needed (cache-first)

    • This combination makes the entire lifecycle fast while ensuring data durability

    • Multiple conserver instances process in parallel while sharing the same storage

    • AI assistants get fast access to all conversation data through the MCP server

    The result is a system that can handle high-volume creation, complex processing, and fast queries, all while maintaining data consistency and durability.

    storages:
      supabase:
        module: storage.supabase
        options:
          url: ${SUPABASE_URL}
          anon_key: ${SUPABASE_ANON_KEY}
          redis_url: ${REDIS_URL}
          cache_ttl: 3600
    SUPABASE_URL=https://your-project.supabase.co
    SUPABASE_ANON_KEY=your-key
    REDIS_URL=redis://localhost:6379
    VCON_REDIS_EXPIRY=3600
    - Stores and retrieves conversation data. This layer uses Supabase and PostgreSQL with extensions for vector search.

    Requests flow from top to bottom. An AI assistant sends a request to the MCP Server Layer. That layer processes it and passes it to the Business Logic Layer. The Business Logic Layer validates and processes the request, then uses the Database Layer to store or retrieve data. Responses flow back up the same path.

    The MCP Server Layer

    The MCP Server Layer is the interface between AI assistants and the server. It speaks the Model Context Protocol, which is a standard way for assistants to interact with external systems.

    Tools, Resources, and Prompts

    The server exposes three types of interfaces:

    Tools are actions the assistant can perform. When you ask the assistant to create a conversation or search for something, it uses a tool. Each tool has a name, description, input parameters, and output format. The assistant reads these definitions and knows how to use each tool.

    Resources are data the assistant can read. Resources use URI paths, similar to URLs. For example, a resource might be vcon://uuid/abc123 to access a specific conversation, or vcon://uuid/abc123/parties to get just the participant information. Resources are read-only, which keeps them safe.

    Prompts are guidance templates that help the assistant work effectively. They explain how to structure queries, what information to include, and best practices. The assistant uses prompts to understand how to accomplish tasks correctly.

    Request Handling

    When an assistant sends a request, here is what happens:

    1. The request arrives through the MCP protocol, either via standard input/output or HTTP

    2. The server parses the JSON-RPC message to understand what the assistant wants

    3. The server identifies which tool, resource, or prompt is being requested

    4. The server prepares to process the request

    5. Plugin hooks can run at this point to modify or intercept the request

    6. The request moves to the Business Logic Layer for processing

    7. The response comes back from the Business Logic Layer

    8. Plugin hooks can run again to modify the response

    9. The response is formatted as an MCP protocol message

    10. The response is sent back to the assistant

    This flow ensures requests are handled consistently and provides opportunities for plugins to add functionality.

    The Business Logic Layer

    The Business Logic Layer contains the core functionality of the server. It handles validation, queries, and extensions.

    The Query Engine

    The query engine handles all database operations. It knows how to create, read, update, and delete conversations. It also handles search operations, component management, and tag operations.

    The query engine is designed to be efficient. It uses database transactions for operations that involve multiple steps, ensuring data consistency. When creating a conversation, for example, it might need to insert records into multiple tables. Using a transaction means either all the inserts succeed or none of them do, preventing partial data.

    The engine also normalizes data when storing it. Normalized data is organized into separate tables with relationships between them. This makes queries efficient and prevents data duplication. When returning data to clients, the engine reconstructs complete conversation objects from the normalized data.

    The Validation Engine

    The validation engine ensures all conversation data follows the IETF vCon standard before it is stored. This prevents invalid data from entering the database and ensures compatibility with other systems.

    Validation checks include:

    • Version numbers must match the current vCon specification version

    • UUIDs must be in the correct format

    • At least one participant must exist

    • Dialog types must be valid

    • Analysis entries must include required fields like vendor information

    • References between components must be valid (for example, a dialog entry cannot reference a participant that does not exist)

    • Encoding values must be valid

    • Dates must be in ISO 8601 format

    If validation fails, the server returns a clear error message explaining what is wrong. This helps users fix their data before trying again.

    The Plugin System

    The plugin system allows extending the server without modifying its core code. Plugins can add new tools, resources, and prompts. They can also intercept operations at various points in the request lifecycle.

    Plugins use hooks that fire at specific times:

    • Before an operation starts, plugins can modify the request

    • After an operation completes, plugins can modify the response

    • Plugins can add logging, access control, data transformation, or other functionality

    For example, a privacy plugin might intercept read operations and remove sensitive information before returning data. An audit plugin might log all operations for compliance tracking. A compliance plugin might check operations against regulatory requirements.

    The plugin system loads plugins when the server starts. Plugins register their hooks, tools, resources, and prompts. When requests come in, the server calls the appropriate hooks and tools from all loaded plugins.

    The Database Layer

    The Database Layer stores conversation data in Supabase, which provides PostgreSQL with additional features.

    Normalized Schema Design

    The database uses a normalized schema, which means data is organized into separate tables with relationships between them. This is different from storing each conversation as a single JSON document.

    The main tables are:

    • vcons - Stores the main conversation record with metadata

    • parties - Stores participants, with a reference to the conversation

    • dialog - Stores conversation content, with references to the conversation and participants

    • analysis - Stores AI analysis results, with references to the conversation and dialog entries

    • attachments - Stores files and documents, with references to the conversation

    • groups - Stores group information for multi-party conversations

    This design has several benefits:

    • Efficient queries. You can search for conversations by participant without loading all conversation data

    • Easy to update. You can add a new dialog entry without touching other parts of the conversation

    • Scalable. The database can handle millions of conversations efficiently

    • Proper constraints. Foreign keys ensure data relationships are valid

    When returning data to clients, the query engine joins these tables together to reconstruct complete conversation objects.

    Search Architecture

    The server supports four types of search, each using different database features:

    Metadata search filters by subject, participant, or dates. It uses B-tree indexes, which are fast for exact matches and range queries. This is the fastest search method, typically returning results in under 100 milliseconds.

    Keyword search looks for words within conversation content. It uses GIN indexes with trigram matching, which allows it to find words even with minor typos. It searches through dialog text, analysis results, and participant information. This method typically takes a few hundred milliseconds.

    Semantic search finds conversations by meaning using AI embeddings. Conversations are converted into vectors, which are mathematical representations of their meaning. The search converts your query into a vector, then finds conversations with similar vectors. It uses an HNSW index, which is optimized for vector similarity searches. This method typically takes one to two seconds.

    Hybrid search combines keyword and semantic search. It runs both searches and merges the results, ranking them based on a weighted combination of both scores. You can control how much weight each method has. This method provides the best of both approaches but takes longer, typically two to three seconds.

    Each search method is implemented as a database function that runs on the database server. This keeps search logic close to the data, which improves performance.

    Tag Storage

    Tags are stored as a special type of attachment. This keeps tags within the vCon format while allowing efficient searching.

    The server maintains a materialized view that extracts tags from attachments and indexes them. A materialized view is a pre-computed query result that is stored in the database and refreshed periodically. This makes tag searches very fast without requiring schema changes when you add new tags.

    When you search by tags, the server uses this materialized view to find conversations quickly. When you add or update tags, the materialized view is updated automatically.

    Caching with Redis

    The server supports optional Redis caching. Redis is an in-memory data store that is much faster than database queries. When enabled, the server checks Redis first before querying the database.

    If data is found in Redis, it returns immediately. If not, it queries the database, stores the result in Redis for future requests, and then returns it. This can make frequently accessed conversations load 20 to 50 times faster.

    Redis caches have expiration times, so cached data does not become stale. When conversations are updated, the cache is cleared for those conversations, ensuring you always get current data.

    Request Flow Examples

    Let us walk through two examples to see how requests flow through the system.

    Creating a Conversation

    You ask the assistant: "Create a vCon for a support call."

    1. The assistant calls the create_vcon tool with conversation data.

    2. The MCP Server Layer receives the request and identifies it as a tool call.

    3. Plugin hooks can run at this point. For example, a plugin might add default tags or metadata.

    4. The Business Logic Layer receives the request. The validation engine checks that all required fields are present and valid.

    5. If validation passes, the query engine starts a database transaction.

    6. The query engine inserts records:

      • First, it inserts the main conversation record into the vcons table

      • Then it inserts participant records into the parties table

    7. The transaction commits, ensuring all inserts succeed together.

    8. Plugin hooks run again. For example, an audit plugin might log the creation, or a webhook plugin might notify another system.

    9. The response is formatted and sent back to the assistant, including the new conversation's UUID.

    10. The assistant receives the response and confirms the conversation was created.

    Semantic Search

    You ask the assistant: "Find frustrated customers from last week."

    1. The assistant calls the search_vcons_semantic tool with a query and date range.

    2. The MCP Server Layer receives the request.

    3. Plugin hooks might modify the search criteria. For example, a multi-tenant plugin might add filters to restrict results to your organization.

    4. The Business Logic Layer receives the request. It converts your query text into an embedding vector using an AI service like OpenAI.

    5. The query engine calls a database function that performs the semantic search. The function:

      • Uses the HNSW index to find conversations with similar embeddings

      • Filters by your date range

      • Applies any tag filters you specified

    6. The query engine reconstructs complete conversation objects by joining the search results with related tables.

    7. Plugin hooks might filter the results. For example, a privacy plugin might remove sensitive conversations before returning them.

    8. The response is formatted and sent back to the assistant with search results.

    9. The assistant receives the results and can analyze or present them to you.

    Performance Considerations

    The server is designed for performance at multiple levels.

    Database Optimization

    The database uses indexes strategically. B-tree indexes on UUIDs and dates make lookups by identifier or date range fast. GIN indexes on text fields enable fast full-text search. HNSW indexes on embeddings enable fast semantic search.

    The server also uses materialized views for frequently accessed data like tags. These views are pre-computed and refreshed periodically, avoiding expensive computations on every query.

    Query patterns are optimized. The server uses prepared statements, which are faster than building queries from strings. It batches operations where possible, reducing the number of database round trips.

    Memory Management

    The server limits result set sizes to prevent memory issues. Large searches return paginated results. The server can stream large responses instead of loading everything into memory at once.

    Plugin resources are cleaned up properly. When plugins are unloaded or the server shuts down, resources are released to prevent memory leaks.

    Scalability

    The server design supports scaling in multiple ways:

    Horizontal scaling means running multiple server instances. Since the server is stateless (it does not store session information), you can run multiple instances behind a load balancer. Requests can be distributed across instances.

    Vertical scaling means increasing the resources available to a single instance. You can add more memory, faster CPUs, or faster database connections.

    Read replicas allow distributing read queries across multiple database copies. Write operations go to the main database, while read operations can use replicas, reducing load on the primary database.

    Type Safety and Validation

    The server is built with TypeScript, which provides compile-time type checking. This means many errors are caught before the code runs. Type definitions match the IETF vCon specification exactly, ensuring the code implements the standard correctly.

    Runtime validation uses Zod, which validates data when it arrives. Even if data comes from an external source that might not follow types correctly, Zod ensures it matches the expected structure before processing.

    The combination of TypeScript types and Zod validation provides both compile-time safety and runtime safety, preventing many classes of bugs.

    Extensibility Through Plugins

    The plugin system allows the server to be extended without modifying core code. This keeps the core simple and focused, while allowing specific needs to be addressed through plugins.

    Plugins can be developed independently and loaded at runtime. They can add functionality like compliance checking, privacy controls, integrations with other systems, or custom analytics.

    The plugin interface is well-defined, so plugins work reliably. As long as a plugin implements the interface correctly, it will work with the server regardless of when it was developed.

    Security Architecture

    Security is handled at multiple levels:

    Authentication ensures only authorized systems can access the server. Supabase provides Row Level Security policies that restrict access based on user identity. API keys validate that requests come from authorized sources.

    Authorization controls what authenticated users can do. Row Level Security policies define which records users can access. Plugins can add additional authorization checks.

    Data protection includes encryption at rest in the database and encryption in transit over the network. Plugins can add redaction to hide sensitive information before returning data.

    The server itself does not store sensitive authentication information. All authentication is handled by Supabase, which is designed for secure data storage.

    Conclusion

    The vCon MCP Server's architecture balances simplicity, performance, and extensibility. The layered design separates concerns, making the code easier to understand and maintain. The normalized database schema ensures efficient queries and scalability. The plugin system allows extending functionality without modifying core code.

    This architecture supports the server's goals of providing reliable conversation data management while remaining flexible enough to meet diverse needs. The next post in this series covers business cases and real-world use cases for the server.

    Gets a list of vCon UUIDs

    get

    Enables pagination of vCon UUIDs. Use the page and size parameters to paginate the results. Can also filter by date with the since and until parameters.

    Query parameters
    pageintegerOptionalDefault: 1
    sizeintegerOptionalDefault: 50
    sincestring · date-timeOptional
    untilstring · date-timeOptional
    Responses
    200

    Successful Response

    application/json
    Responsestring[]
    422

    Validation Error

    application/json
    get
    /vcon

    Inserts a vCon into the database

    post

    How to insert a vCon into the database.

    Body
    vconstringRequired
    uuidstring · uuidRequired
    created_atany ofRequired
    integerOptional
    or
    stringOptional
    or
    string · date-timeOptional
    subjectany ofOptional
    stringOptional
    or
    nullOptional
    redactedobject · RedactedOptionalDefault: {}
    appendedany ofOptional
    objectOptional
    or
    nullOptional
    groupobject[]OptionalDefault: []
    partiesobject[]OptionalDefault: []
    dialogobject[]OptionalDefault: []
    analysisobject[]OptionalDefault: []
    attachmentsobject[]OptionalDefault: []
    metaany ofOptionalDefault: {}
    objectOptional
    or
    nullOptional
    Responses
    200

    Successful Response

    application/json
    422

    Validation Error

    application/json
    post
    /vcon

    Gets a particular vCon by UUID

    get

    How to get a particular vCon by UUID

    Path parameters
    vcon_uuidstring · uuidRequired
    Responses
    200

    Successful Response

    application/json
    422

    Validation Error

    application/json
    get
    /vcon/{vcon_uuid}

    Deletes a particular vCon by UUID

    delete

    How to remove a vCon from the conserver.

    Path parameters
    vcon_uuidstring · uuidRequired
    Responses
    204

    Successful Response

    422

    Validation Error

    application/json
    delete
    /vcon/{vcon_uuid}

    No content

    Inserts a vCon UUID into one or more chains

    post

    Inserts a vCon UUID into one or more chains.

    Query parameters
    ingress_liststringRequired
    Bodystring[]
    string[]Optional
    Responses
    204

    Successful Response

    422

    Validation Error

    application/json
    post
    /vcon/ingress

    No content

    Removes one or more vCon UUIDs from the output of a chain (egress)

    get

    Removes one or more vCon UUIDs from the output of a chain (egress)

    Query parameters
    egress_liststringRequired
    limitanyOptionalDefault: 1
    Responses
    204

    Successful Response

    422

    Validation Error

    application/json
    get
    /vcon/egress

    No content

    Returns the number of vCons at the end of a chain

    get

    Returns the number of vCons at the end of a chain.

    Query parameters
    egress_liststringRequired
    Responses
    204

    Successful Response

    422

    Validation Error

    application/json
    get
    /vcon/count

    No content

    Returns the config file for the conserver

    get

    Returns the config file for the conserver

    Responses
    200

    Successful Response

    application/json
    Responseany
    get
    /config
    200

    Successful Response

    No content

    Updates the config file for the conserver

    post

    Updates the config file for the conserver

    Query parameters
    update_file_nameanyOptional
    Body
    object · ConfigOptional
    Responses
    204

    Successful Response

    422

    Validation Error

    application/json
    post
    /config

    No content

    Clears the config file for the conserver

    delete

    Clears the config file for the conserver

    Responses
    204

    Successful Response

    delete
    /config
    204

    Successful Response

    No content

    Reprocess the dead letter queue

    post

    Move the dead letter queue vcons back to the ingress chain

    Query parameters
    ingress_liststringRequired
    Responses
    200

    Successful Response

    application/json
    Responseany
    422

    Validation Error

    application/json
    post
    /dlq/reprocess

    No content

    Get Vcons list from the dead letter queue

    get

    Get Vcons list from the dead letter queue, returns array of vcons.

    Query parameters
    ingress_liststringRequired
    Responses
    200

    Successful Response

    application/json
    Responseany
    422

    Validation Error

    application/json
    get
    /dlq

    No content

    Forces a reset of the vcon search list

    get

    Forces a reset of the vcon search list, returns the number of vCons indexed.

    Responses
    200

    Successful Response

    application/json
    Responseany
    get
    /index_vcons
    200

    Successful Response

    No content

    Search vCons based on various parameters

    get

    Search for vCons using personal identifiers and metadata.

    Query parameters
    telany ofOptional

    Phone number to search for

    stringOptional
    or
    nullOptional
    mailtoany ofOptional

    Email address to search for

    stringOptional
    or
    nullOptional
    nameany ofOptional

    Name of the party to search for

    stringOptional
    or
    nullOptional
    Responses
    200

    Successful Response

    application/json
    422

    Validation Error

    application/json

    Coding Prompt Cheat Sheet

    For when you need to tell cursor or replit what a vCon is in a prompt...

    vCon (Virtual Conversation) Standard - LLM Context

    What is a vCon?

    A vCon is a standardized JSON container for storing and exchanging real-time human conversation data. It supports multiple communication types: phone calls, video conferences, SMS, MMS, emails, web chat, and more. vCons enable consistent storage, analysis, and interchange of conversational data across different platforms and services.

    GET /config HTTP/1.1
    Host: 
    Accept: */*
    
    DELETE /config HTTP/1.1
    Host: 
    Accept: */*
    
    GET /index_vcons HTTP/1.1
    Host: 
    Accept: */*
    
    If dialog content is provided, it inserts dialog records
  • If analysis results are provided, it inserts analysis records

  • Ranks results by similarity

  • Returns the top matches

  • Logo
    Logo
    get
    /vcons/search

    Core Structure

    Every vCon has exactly 5 main sections:

    1. metadata - conversation context and identifiers

    2. parties - participant information

    3. dialog - actual conversation content

    4. analysis - derived insights (transcripts, sentiment, etc.)

    5. attachments - supplemental files

    vCon States

    • unsigned - initial/intermediate state during collection

    • signed - verified with JWS digital signature for immutability

    • encrypted - secured with JWE for sensitive data

    Complete JSON Schema

    Top-Level vCon Object Properties

    Party Object Properties

    Dialog Object Properties

    Analysis Object Properties

    Attachment Object Properties

    Redacted Object Properties

    Appended Object Properties

    Group Object Properties

    Party History Object Properties (for Dialog.party_history)

    Security Features

    • JWS Signing - ensures integrity and authenticity using RS256 (recommended)

    • JWE Encryption - protects sensitive content using RSA-OAEP + A256CBC-HS512 (recommended)

    • Content Hashing - SHA-512 hashes for external file integrity (mandatory for external refs)

    • Versioning - maintains history of changes and redactions via uuid references

    Signed vCon Structure (JWS)

    Encrypted vCon Structure (JWE)

    Common Media Types

    Dialog

    • text/plain - plain text messages

    • audio/x-wav - WAV audio files

    • audio/x-mp3 - MP3 audio files

    • audio/x-mp4 - MP4 audio files

    • audio/ogg - OGG audio files

    • video/x-mp4 - MP4 video files

    • video/ogg - OGG video files

    • multipart/mixed - multipart content (emails)

    Content Encoding Options

    • base64url - Base64url encoded binary data

    • json - Valid JSON object

    • none - Valid JSON string, no encoding needed

    Implementation Guidelines

    • Follow JSON schema strictly for compliance

    • Use proper timestamps (RFC3339/ISO 8601 format)

    • Ensure UUIDs are globally unique (prefer version 8 with domain-based generation)

    • Implement proper signing/encryption for production use

    • Maintain referential integrity between sections

    • Always use HTTPS for external file references

    • Validate content hashes for external files using SHA-512

    Python Development Notes

    • Use uuid library for generating UUIDs (version 8 recommended)

    • datetime.isoformat() for RFC3339 timestamps

    • json module for serialization/deserialization

    • cryptography library for JWS/JWE operations

    • hashlib for SHA-512 content hash generation

    • Validate against vCon JSON schema before processing

    • Use requests with SSL verification for external file retrieval

    Key Considerations

    • vCons can reference previous versions (redaction/append history via uuid)

    • Media content can be embedded (body + encoding) or referenced externally (url + content_hash)

    • Privacy and compliance requirements vary by jurisdiction

    • Large media files should typically be stored as external references

    • Ensure proper escaping of JSON content in dialog sections

    • Support both single-channel and multi-channel audio recordings

    • Handle participant join/leave events in party_history for complex conversations

    • Maintain chain of custody through signing and encryption across security domains

    Validation Rules

    • At most one of: redacted, appended, or group parameters in top-level object

    • Dialog objects of type "incomplete" or "transfer" MUST NOT have body/url content

    • Dialog objects of other types SHOULD have body+encoding OR url+content_hash

    • External references (url) MUST include content_hash for integrity

    • Signed vCons MUST include x5c OR x5u in header for certificate chain

    • Party indices in dialog.parties must reference valid parties array elements

    • Dialog indices in analysis.dialog must reference valid dialog array elements

    UUID Generation (Version 8 Recommended)

    GET /vcons/search HTTP/1.1
    Host: 
    Accept: */*
    
    [
      {
        "uuid": "text",
        "created_at": "2025-12-10T10:22:20.954Z",
        "updated_at": "2025-12-10T10:22:20.954Z",
        "subject": "text",
        "parties": [
          {}
        ]
      }
    ]
    GET /vcon HTTP/1.1
    Host: 
    Accept: */*
    
    [
      "text"
    ]
    POST /vcon HTTP/1.1
    Host: 
    Content-Type: application/json
    Accept: */*
    Content-Length: 208
    
    {
      "vcon": "text",
      "uuid": "123e4567-e89b-12d3-a456-426614174000",
      "created_at": 1,
      "subject": "text",
      "redacted": {},
      "appended": {},
      "group": [
        {}
      ],
      "parties": [
        {}
      ],
      "dialog": [
        {}
      ],
      "analysis": [
        {}
      ],
      "attachments": [
        {}
      ],
      "meta": {}
    }
    {
      "vcon": "text",
      "uuid": "123e4567-e89b-12d3-a456-426614174000",
      "created_at": 1,
      "subject": "text",
      "redacted": {},
      "appended": {},
      "group": [
        {}
      ],
      "parties": [
        {}
      ],
      "dialog": [
        {}
      ],
      "analysis": [
        {}
      ],
      "attachments": [
        {}
      ],
      "meta": {}
    }
    GET /vcon/{vcon_uuid} HTTP/1.1
    Host: 
    Accept: */*
    
    {
      "vcon": "text",
      "uuid": "123e4567-e89b-12d3-a456-426614174000",
      "created_at": 1,
      "subject": "text",
      "redacted": {},
      "appended": {},
      "group": [
        {}
      ],
      "parties": [
        {}
      ],
      "dialog": [
        {}
      ],
      "analysis": [
        {}
      ],
      "attachments": [
        {}
      ],
      "meta": {}
    }
    DELETE /vcon/{vcon_uuid} HTTP/1.1
    Host: 
    Accept: */*
    
    POST /vcon/ingress?ingress_list=text HTTP/1.1
    Host: 
    Content-Type: application/json
    Accept: */*
    Content-Length: 8
    
    [
      "text"
    ]
    GET /vcon/egress?egress_list=text HTTP/1.1
    Host: 
    Accept: */*
    
    GET /vcon/count?egress_list=text HTTP/1.1
    Host: 
    Accept: */*
    
    POST /config HTTP/1.1
    Host: 
    Content-Type: application/json
    Accept: */*
    Content-Length: 2
    
    {}
    POST /dlq/reprocess?ingress_list=text HTTP/1.1
    Host: 
    Accept: */*
    
    GET /dlq?ingress_list=text HTTP/1.1
    Host: 
    Accept: */*
    
    {
      "vcon": "0.0.2",                    // REQUIRED: syntax version
      "uuid": "string",                   // REQUIRED: globally unique identifier 
      "created_at": "Date",               // REQUIRED: creation timestamp (RFC3339)
      "updated_at": "Date",               // OPTIONAL: last modification timestamp
      "subject": "string",                // OPTIONAL: conversation topic/subject
      "parties": [],                      // REQUIRED: array of Party objects
      "dialog": [],                       // OPTIONAL: array of Dialog objects
      "analysis": [],                     // OPTIONAL: array of Analysis objects  
      "attachments": [],                  // OPTIONAL: array of Attachment objects
      "redacted": {},                     // OPTIONAL: Redacted object (mutually exclusive with appended/group)
      "appended": {},                     // OPTIONAL: Appended object (mutually exclusive with redacted/group)
      "group": []                         // OPTIONAL: array of Group objects (mutually exclusive with redacted/appended)
    }
    {
      "tel": "string",                    // OPTIONAL: telephone number (E.164 format preferred)
      "stir": "string",                   // OPTIONAL: STIR PASSporT in JWS Compact form
      "mailto": "string",                 // OPTIONAL: email address
      "name": "string",                   // OPTIONAL: participant name
      "validation": "string",             // OPTIONAL: identity validation method used
      "jcard": "object",                  // OPTIONAL: jCard object for contact info
      "gmlpos": "string",                 // OPTIONAL: GML position (lat/long)
      "civicaddress": {                   // OPTIONAL: civic address object
        "country": "string",              // OPTIONAL: country code
        "a1": "string",                   // OPTIONAL: national subdivision (state/province)
        "a2": "string",                   // OPTIONAL: county/parish/district
        "a3": "string",                   // OPTIONAL: city/township
        "a4": "string",                   // OPTIONAL: city division/borough
        "a5": "string",                   // OPTIONAL: neighborhood/block
        "a6": "string",                   // OPTIONAL: street/group of streets
        "prd": "string",                  // OPTIONAL: leading street direction
        "pod": "string",                  // OPTIONAL: trailing street suffix
        "sts": "string",                  // OPTIONAL: street suffix
        "hno": "string",                  // OPTIONAL: house number
        "hns": "string",                  // OPTIONAL: house number suffix
        "lmk": "string",                  // OPTIONAL: landmark
        "loc": "string",                  // OPTIONAL: additional location info
        "flr": "string",                  // OPTIONAL: floor
        "nam": "string",                  // OPTIONAL: name/description
        "pc": "string"                    // OPTIONAL: postal code
      },
      "timezone": "string",               // OPTIONAL: timezone identifier
      "uuid": "string",                   // OPTIONAL: unique participant identifier
      "role": "string",                   // OPTIONAL: participant role (agent, customer, supervisor, etc.)
      "contact_list": "string"            // OPTIONAL: reference to contact list
    }
    {
      "type": "string",                   // REQUIRED: "recording", "text", "transfer", or "incomplete"
      "start": "Date",                    // REQUIRED: start time (RFC3339)
      "duration": "number",               // OPTIONAL: duration in seconds (UnsignedInt or UnsignedFloat)
      "parties": [],                      // REQUIRED: array of party indices or arrays for multi-channel
      "originator": "number",             // OPTIONAL: index of originating party (if not first in parties)
      "mediatype": "string",              // OPTIONAL: MIME type (required for inline, optional if in HTTP header)
      "filename": "string",               // OPTIONAL: original filename
      
      // Content (for types other than "incomplete" and "transfer")
      "body": "string",                   // OPTIONAL: inline content (mutually exclusive with url)
      "encoding": "string",               // REQUIRED with body: "base64url", "json", or "none"
      "url": "string",                    // OPTIONAL: external reference (mutually exclusive with body)
      "content_hash": "string|string[]",  // REQUIRED with url: SHA-512 hash for integrity
      
      // Additional properties
      "disposition": "string",            // REQUIRED for "incomplete" type: reason for failure
      "party_history": [],               // OPTIONAL: array of party join/leave events
      "campaign": "string",               // OPTIONAL: campaign identifier
      "interaction_type": "string",       // OPTIONAL: type of interaction
      "interaction_id": "string",         // OPTIONAL: interaction identifier
      "skill": "string",                  // OPTIONAL: required skill for handling
      "application": "string",            // OPTIONAL: application/platform used
      "message_id": "string",             // OPTIONAL: unique message identifier
      
      // Transfer-specific properties (only for "transfer" type)
      "transferee": "number",             // Party index for transferee role
      "transferor": "number",             // Party index for transferor role  
      "transfer_target": "number",        // Party index for transfer target role
      "original": "number",               // Dialog index for original conversation
      "consultation": "number",           // Dialog index for consultation (optional)
      "target_dialog": "number"           // Dialog index for target conversation
    }
    {
      "type": "string",                   // REQUIRED: "summary", "transcript", "translation", "sentiment", "tts"
      "dialog": "number|number[]",        // OPTIONAL: index(es) of related dialog objects
      "mediatype": "string",              // OPTIONAL: MIME type of analysis data
      "filename": "string",               // OPTIONAL: filename for analysis data
      "vendor": "string",                 // OPTIONAL: vendor/product name that generated analysis
      "product": "string",                // OPTIONAL: specific product name
      "schema": "string",                 // OPTIONAL: data format/schema identifier
      
      // Content
      "body": "string",                   // OPTIONAL: inline analysis data (mutually exclusive with url)
      "encoding": "string",               // REQUIRED with body: encoding method
      "url": "string",                    // OPTIONAL: external reference (mutually exclusive with body)
      "content_hash": "string|string[]"   // REQUIRED with url: integrity hash
    }
    {
      "type": "string",                   // OPTIONAL: semantic type of attachment
      "purpose": "string",                // OPTIONAL: purpose of attachment
      "start": "Date",                    // REQUIRED: timestamp when attachment was exchanged
      "party": "number",                  // REQUIRED: index of party who contributed attachment
      "mediatype": "string",              // OPTIONAL: MIME type
      "filename": "string",               // OPTIONAL: original filename
      "dialog": "number",                 // REQUIRED: index of related dialog
      
      // Content
      "body": "string",                   // OPTIONAL: inline attachment data (mutually exclusive with url)
      "encoding": "string",               // REQUIRED with body: encoding method
      "url": "string",                    // OPTIONAL: external reference (mutually exclusive with body)
      "content_hash": "string|string[]"   // REQUIRED with url: integrity hash
    }
    {
      "uuid": "string",                   // REQUIRED: UUID of unredacted version
      "type": "string",                   // REQUIRED: type of redaction performed
      "body": "string",                   // OPTIONAL: inline unredacted vCon (encrypted)
      "encoding": "string",               // REQUIRED with body: encoding method
      "url": "string",                    // OPTIONAL: external reference to unredacted vCon
      "content_hash": "string|string[]"   // REQUIRED with url: integrity hash
    }
    {
      "uuid": "string",                   // OPTIONAL: UUID of original vCon version
      "body": "string",                   // OPTIONAL: inline original vCon
      "encoding": "string",               // REQUIRED with body: encoding method
      "url": "string",                    // OPTIONAL: external reference to original vCon
      "content_hash": "string|string[]"   // OPTIONAL with url: integrity hash
    }
    {
      "uuid": "string",                   // REQUIRED: UUID of vCon to aggregate
      "body": "string",                   // OPTIONAL: inline vCon (JSON form)
      "encoding": "string",               // REQUIRED with body: must be "json"
      "url": "string",                    // OPTIONAL: external reference to vCon
      "content_hash": "string|string[]"   // REQUIRED with url: integrity hash
    }
    {
      "party": "number",                  // REQUIRED: index of party
      "event": "string",                  // REQUIRED: "join", "drop", "hold", "unhold", "mute", "unmute"
      "time": "Date"                      // REQUIRED: timestamp of event
    }
    {
      "payload": "string",                // Base64url encoded unsigned vCon
      "signatures": [{                    // Array of signature objects
        "header": {                       // Unprotected header
          "alg": "RS256",                 // SHOULD be RS256
          "x5c": ["string"],              // REQUIRED: certificate chain OR x5u
          "x5u": "string",                // REQUIRED: cert chain URL OR x5c
          "uuid": "string"                // SHOULD be provided: vCon UUID for convenience
        },
        "protected": "string",            // Base64url encoded protected header
        "signature": "string"             // Base64url encoded signature
      }]
    }
    {
      "unprotected": {                    // Unprotected header
        "cty": "application/vcon+json",   // SHOULD be application/vcon+json
        "enc": "A256CBC-HS512",           // SHOULD be A256CBC-HS512
        "uuid": "string"                  // SHOULD be provided: vCon UUID
      },
      "recipients": [{                    // Array of recipient objects
        "header": {                       // Per-recipient header
          "alg": "RSA-OAEP"               // SHOULD be RSA-OAEP
        },
        "encrypted_key": "string"         // Base64url encoded encrypted key
      }],
      "iv": "string",                     // Base64url encoded initialization vector
      "ciphertext": "string",             // Base64url encoded encrypted signed vCon
      "tag": "string"                     // Base64url encoded authentication tag
    }
    # Recommended UUID generation approach
    import hashlib
    import uuid
    from datetime import datetime
    
    def generate_vcon_uuid(domain="example.com"):
        """Generate version 8 UUID for vCon with domain-based uniqueness"""
        timestamp = int(datetime.utcnow().timestamp() * 1000)  # milliseconds
        domain_hash = hashlib.sha1(domain.encode()).digest()[:8]  # 62 bits
        
        # Construct version 8 UUID (implementation details vary)
        # Use standard uuid library with custom generation
        return str(uuid.uuid4())  # Fallback to uuid4 if version 8 not available

    Conserver Tracers

    Tracer Functionality in vCon Server

    Overview

    Tracer functionality in the vCon Server provides a powerful mechanism for observability, auditing, and compliance tracking as vCons (virtual conversations) flow through processing chains. Unlike processing links that transform or analyze vCon data, tracers are non-intrusive monitoring components that observe and record data flow without modifying the vCon content itself.

    Key Concepts

    What are Tracers?

    Tracers are side-effect modules that execute at specific points in the vCon processing pipeline to:

    • Monitor data flow between processing links

    • Create audit trails for compliance and security

    • Track provenance and data lineage

    • Generate observability metrics without affecting the main processing flow

    Tracers vs. Links

    Execution Model

    Tracer Invocation Points

    Tracers are executed at three critical points in the processing pipeline:

    1. Before First Link (link_index = -1)

    • Executes when a vCon enters the processing chain

    • Records initial state and metadata

    1. After Each Link (link_index = 0, 1, 2, ...)

    • Executes after each processing link completes

    • Captures data transformations and flow

    1. Chain Completion

    • Executes when the entire processing chain finishes

    • Records final state and completion metrics

    Execution Flow

    Available Tracer Modules

    1. JLINC Zero-Knowledge Auditing

    The JLINC tracer provides cryptographic signing and zero-knowledge audit capabilities for tamper-proof data provenance.

    Features

    • Cryptographic Signing: Creates tamper-proof signatures for vCon data

    • Zero-Knowledge Auditing: Enables secure third-party auditing without exposing sensitive data

    • Entity Management: Automatically creates and manages JLINC entities for each processing stage

    • Data Hashing: Optionally hash vCon data for privacy-preserving audit trails

    Configuration

    How JLINC Tracer Works

    Entity Creation: Creates JLINC entities for each processing stage

    • System entity: {system_prefix}-system@{domain}

    • Link entities: {system_prefix}-{link_name}@{domain}

    Event Processing: For each vCon transition:

    • Retrieves vCon data from Redis

    • Creates sender/recipient entities based on link context

    • Optionally hashes vCon data for privacy

    • Sends event to JLINC API for cryptographic signing

    Audit Trail: Creates immutable audit records with:

    • Cryptographic signatures

    • Data hashes (if enabled)

    • Metadata (vCon UUIDs, link information)

    • Timestamps and provenance information

    Configuration

    Basic Tracer Configuration

    Multiple Tracers

    You can configure multiple tracers to run simultaneously:

    Tracer Interface

    Required Function Signature

    All tracer modules must implement a run function with this signature:

    Implementation Example

    Use Cases

    1. Compliance and Auditing

    • GDPR Compliance: Track data processing for privacy regulations

    • SOX Compliance: Audit financial conversation processing

    • HIPAA Compliance: Monitor healthcare conversation handling

    2. Security and Integrity

    • Data Provenance: Track data lineage and transformations

    • Tamper Detection: Cryptographic verification of data integrity

    • Access Logging: Record who accessed what data when

    3. Observability and Monitoring

    • Performance Metrics: Track processing times and throughput

    • Error Tracking: Monitor failures and exceptions

    • Business Metrics: Count conversations, analyze patterns

    4. Data Governance

    • Retention Tracking: Monitor data lifecycle and expiration

    • Data Classification: Track sensitive data handling

    • Cross-Border Transfers: Monitor international data flows

    Best Practices

    1. Non-Blocking Design

    • Tracers should never block the main processing flow

    • Handle errors gracefully without affecting vCon processing

    • Use asynchronous operations where possible

    2. Performance Considerations

    • Keep tracer execution time minimal

    • Cache frequently accessed data

    • Use efficient data serialization

    3. Error Handling

    • Log errors but don't raise exceptions

    • Return boolean status for success/failure

    • Implement retry logic for external API calls

    4. Privacy and Security

    • Hash or encrypt sensitive data before external transmission

    • Follow data minimization principles

    • Implement proper authentication and authorization

    5. Configuration Management

    • Provide sensible defaults

    • Validate configuration options

    • Support environment-specific settings

    Monitoring and Debugging

    Logging

    Tracers automatically log their execution with structured logging:

    Metrics

    Tracer execution is automatically tracked with:

    • Processing time per tracer

    • Success/failure rates

    • vCon throughput metrics

    Debugging

    • Enable debug logging for detailed tracer execution

    • Use tracer-specific configuration for testing

    • Monitor external API responses and errors

    Future Extensions

    The tracer system is designed to be extensible. Potential future tracer modules could include:

    • DataTrails Integration: Blockchain-based audit trails

    • SIEM Integration: Security information and event management

    • Custom Analytics: Business intelligence and reporting

    • Data Loss Prevention: Monitor for sensitive data exposure

    Conclusion

    Tracer functionality provides a powerful, non-intrusive way to add observability, compliance, and security monitoring to vCon processing pipelines. By executing alongside the main processing flow without affecting it, tracers enable comprehensive data governance and audit capabilities while maintaining system performance and reliability.The modular design allows for easy extension with custom tracer implementations, making it possible to integrate with any external system or compliance framework while maintaining the core principle of non-interference with vCon processing.

    LLM Guide: Creating vCon Adapters

    This guide provides essential information for Large Language Models tasked with generating vCon adapter code. Follow these patterns and requirements when creating adapters that convert conversation da

    Core Requirements

    Essential Imports

    Always include these imports in vCon adapter code:

    vCon Lifecycle Management using SCITT

    A Privacy-First Approach to Conversation Data Management

    Executive Summary

    Organizations across industries are grappling with an unprecedented challenge: how to extract value from conversational data while maintaining strict privacy compliance and earning consumer trust. Every day, billions of conversations occur through phone calls, video conferences, chat systems, and email exchanges, generating valuable insights for customer service, artificial intelligence training, and business intelligence. However, this same data contains sensitive personal information subject to increasingly stringent privacy regulations worldwide.

    Traditional approaches to conversation data management create fragmented systems where data, consent records, and compliance information exist in isolated silos. When individuals exercise their privacy rights—requesting access to their data, corrections to inaccurate information, or complete deletion—organizations often struggle to provide comprehensive responses or coordinate actions across multiple systems.

    This whitepaper introduces a revolutionary approach that combines Virtualized Conversations (vCons) with Supply Chain Integrity, Transparency, and Trust (SCITT) protocols to create a comprehensive framework for privacy-first conversation data management. This approach embeds consent management directly into conversation containers, creates immutable audit trails, and enables automated compliance across distributed systems.

    The implications extend far beyond technical implementation. Organizations adopting this framework can transform privacy from a compliance burden into a competitive advantage, building customer trust through unprecedented transparency while enabling responsible innovation in artificial intelligence and data analytics.

    The Privacy Imperative in Conversation Data

    The landscape of privacy regulation has fundamentally shifted over the past decade. The European Union's General Data Protection Regulation (GDPR), California's Consumer Privacy Act (CCPA), and similar laws worldwide have established individual rights that organizations must respect and protect. These regulations are not merely compliance checkboxes; they reflect a societal expectation that individuals should maintain meaningful control over their personal information.

    Conversational data presents unique challenges within this regulatory framework. Unlike static customer records or transaction data, conversations are dynamic, multi-party interactions that may be processed by various systems over time. A single customer service call might be recorded by a telephony system, transcribed by an AI service, analyzed for sentiment by another platform, and used to train machine learning models by a fourth system. Each step in this process requires explicit consent and creates obligations for data protection.

    The complexity multiplies when considering the temporal aspects of consent. Privacy regulations establish that consent is not a one-time agreement but an ongoing relationship that individuals can modify or revoke at any time. When a customer withdraws consent for AI training, every system that has processed their conversation data must respond appropriately. Currently, most organizations lack the infrastructure to identify where data exists, verify the authority of deletion requests, or coordinate responses across multiple systems.

    Furthermore, the global nature of modern business means that conversation data often crosses jurisdictional boundaries, each with distinct privacy requirements. An international company might have customer data processed in European data centers, analyzed by AI systems in North America, and stored in cloud infrastructure spanning multiple continents. Managing consent and compliance across this distributed landscape requires standardized approaches that can operate across different legal frameworks.

    The stakes of privacy compliance extend beyond regulatory fines, though these can be substantial. Organizations face reputational risks, competitive disadvantages, and operational disruptions when privacy incidents occur. More fundamentally, privacy protection has become a consumer expectation and market differentiator. Companies that can demonstrate transparent, responsible data practices gain customer trust and loyalty in an increasingly privacy-conscious marketplace.

    Understanding Virtualized Conversations (vCons)

    Virtualized Conversations, or vCons, represent a paradigm shift in how conversational data is structured, stored, and shared. Rather than treating conversation elements as disparate pieces of information scattered across multiple systems, vCons create standardized containers that keep all related data together in a cohesive, portable format.

    The vCon specification defines a comprehensive framework for conversation representation that accommodates the full spectrum of modern communication modalities. Whether the interaction occurs through traditional telephony, video conferencing platforms, instant messaging, email threads, or emerging communication channels, vCons provide a consistent structure for capturing and organizing the associated data.

    At its core, a vCon contains four primary components, each serving distinct purposes in conversation data management. The parties section identifies all participants in the conversation, including detailed contact information, roles, and relationships. This goes beyond simple name and phone number records to include structured data about participant authority, organizational affiliations, and communication preferences. The dialog section captures the actual conversation content in its original form, whether audio recordings, video files, text transcripts, or multimedia messages. This raw conversational data maintains its integrity and authenticity while being packaged in a standardized format that any compatible system can process.

    The analysis section houses derived insights and transformations of the original conversation data. This might include automatically generated transcripts, sentiment analysis results, topic extraction, speaker identification, or any other algorithmic processing of the conversation content. Importantly, the analysis section maintains clear linkages to the original dialog data, enabling audit trails and verification of analytical accuracy. The attachments section provides space for supplementary materials related to the conversation, such as documents shared during the interaction, consent forms, digital signatures, or other contextual information that enhances understanding of the conversational context.

    Beyond these core components, vCons include comprehensive metadata that enables sophisticated conversation lifecycle management. Unique identifiers ensure that conversations can be tracked and referenced across systems and time periods. Timestamps capture not only when conversations occurred but also when various processing steps were completed. Version control mechanisms support conversation evolution, allowing organizations to update or append information while maintaining historical integrity.

    The standardization aspect of vCons cannot be overstated in its importance. Currently, every communication platform, customer relationship management system, and analytics tool uses proprietary formats for storing conversational data. This fragmentation creates vendor lock-in, complicates data migration, and makes comprehensive privacy management nearly impossible. vCons establish a lingua franca for conversational data that enables interoperability between systems, facilitates vendor independence, and supports comprehensive lifecycle management.

    Moreover, vCons are designed to be both human-readable and machine-processable. The underlying JSON format makes vCon contents accessible to developers and auditors while maintaining compatibility with modern software architectures. This dual accessibility supports both automated processing workflows and manual review processes required for compliance verification.

    Supply Chain Integrity, Transparency, and Trust (SCITT)

    Supply Chain Integrity, Transparency, and Trust (SCITT) represents a breakthrough in creating verifiable, tamper-evident records of digital events and transactions. While originally conceived for supply chain transparency, SCITT's principles and architecture prove remarkably well-suited to privacy and consent management challenges.

    Traditional database systems, even those with strong security controls, operate on a paradigm where records can be modified, updated, or deleted by authorized users. While this flexibility supports operational needs, it creates challenges for compliance and auditing. How can an organization prove to a regulator that consent records haven't been modified after the fact? How can individuals trust that their data deletion requests were actually honored rather than simply marked as deleted in a database that could be reversed?

    SCITT addresses these challenges through an append-only ledger architecture that creates permanent, verifiable records of events. Once an entry is added to a SCITT transparency service, it becomes mathematically impossible to alter or remove that entry without detection. This immutability creates a foundation of trust that benefits all parties involved in privacy and consent management.

    The transparency service operates through a distributed network of nodes that collectively maintain the integrity of the ledger. When an organization wants to record a consent decision, data processing event, or compliance action, they submit a signed statement to the transparency service. The service validates the statement, adds it to the permanent ledger, and returns a cryptographic receipt that proves the statement was recorded at a specific time.

    These receipts serve multiple critical functions in privacy management. For organizations, receipts provide irrefutable proof that they have documented consent decisions, recorded data processing activities, and responded to individual rights requests. For individuals, receipts offer verification that their privacy choices have been officially recorded and cannot be disputed later. For regulators, receipts enable efficient audit processes where compliance can be verified through cryptographic proof rather than extensive document review.

    The cryptographic foundations of SCITT ensure that statements and receipts cannot be forged or manipulated. Digital signatures verify the identity of entities making statements, while hash functions and Merkle tree structures ensure that any attempt to modify historical records would be immediately detectable. This mathematical certainty removes the need for trust relationships between parties, as the protocol itself provides verification mechanisms.

    Importantly, SCITT's transparency doesn't mean that all information becomes publicly visible. The system can operate with privacy-preserving mechanisms where the existence and timing of events are transparent while the content remains confidential to authorized parties. This balance enables public verifiability of compliance processes while protecting sensitive personal information.

    The distributed nature of SCITT also provides resilience against single points of failure or malicious actors. Unlike centralized systems where a single breach or system failure could compromise all records, SCITT's distributed ledger ensures that transparency and verification capabilities remain available even if individual nodes are compromised or become unavailable.

    Integrating vCons with SCITT for Comprehensive Lifecycle Management

    The integration of vCons with SCITT creates a powerful framework that addresses the complete lifecycle of conversational data while maintaining privacy protection and regulatory compliance. This integration operates on multiple levels, from initial consent collection through final data deletion, creating an end-to-end system for responsible conversation data management.

    The lifecycle begins with conversation initiation and consent collection. Modern communication systems can automatically detect when conversations involve personal data and trigger appropriate consent collection mechanisms. Rather than treating consent as a separate, disconnected process, the vCon framework embeds consent information directly into the conversation container through standardized consent attachments. These attachments capture not only the consent decisions but also the context in which consent was given, the specific purposes for which consent was granted, and any limitations or conditions on that consent.

    When consent is collected, the decision is immediately recorded in a SCITT transparency service, creating an immutable timestamp and cryptographic proof of the consent event. This recording includes essential metadata such as the identity of the consenting party, the specific purposes for which consent was granted, any expiration dates or renewal requirements, and references to applicable terms of service or privacy policies. The SCITT receipt provides immediate verification that the consent was properly recorded and cannot be disputed later.

    As conversations are processed and enhanced through various systems, each step in the data lifecycle is documented through additional SCITT entries. When a conversation is transcribed by an AI service, analyzed for sentiment, or used to train machine learning models, these processing events are recorded with references to the original consent authorizations. This creates a comprehensive audit trail that demonstrates compliance with consent limitations and provides visibility into how personal data has been used.

    The embedded nature of consent information in vCons ensures that privacy constraints travel with the data itself. When a vCon is shared between organizations or processing systems, the receiving party can immediately verify the consent status and any limitations on data use. This eliminates the common scenario where data is shared without proper consent verification, as the consent information is integral to the data container itself.

    Consent management through this integrated system supports the full spectrum of individual privacy rights. When someone requests access to their conversation data, the SCITT transparency service provides a complete audit trail of all processing activities, enabling organizations to provide comprehensive responses. When consent is modified or revoked, the change is recorded in SCITT and propagated to all systems holding the relevant vCons, ensuring coordinated compliance across distributed architectures.

    The temporal aspects of consent are particularly well-served by this integration. Privacy regulations often require that consent be renewed periodically, especially for sensitive data processing activities. The vCon consent framework supports configurable verification intervals based on data sensitivity and regulatory requirements. High-sensitivity medical or financial conversations might require daily consent verification, while routine customer service interactions might require weekly or monthly checks. The SCITT transparency service ensures that these verification activities are documented and that any changes in consent status are immediately visible to all relevant systems.

    Perhaps most importantly, this integration provides mechanisms for proving compliance to regulators and auditors. Rather than requiring organizations to produce extensive documentation during privacy audits, regulators can verify compliance through cryptographic proof provided by SCITT receipts. This transforms regulatory compliance from a document-intensive process to an automated verification system that benefits both organizations and oversight bodies.

    Consent Attachments: Embedding Privacy Protection in Data

    The concept of consent attachments represents a fundamental innovation in how privacy protections are implemented in practice. Rather than treating consent as metadata stored separately from personal data, consent attachments embed privacy controls directly into conversation containers, ensuring that consent information travels with data throughout its entire lifecycle.

    Traditional consent management systems create significant operational challenges because consent records are typically stored in centralized databases or identity management systems, separated from the data they govern. This separation creates multiple points of failure in privacy protection. Data might be processed without proper consent verification, shared between systems without privacy constraints, or retained beyond consent expiration dates because the processing systems lack visibility into consent status.

    Consent attachments solve these problems by making privacy protection an integral property of the data itself. Each vCon can contain one or more consent attachments that specify exactly what processing activities are authorized, which parties have granted consent, and what limitations or conditions apply to data use. These attachments use structured metadata formats that both humans and automated systems can understand and enforce.

    The structure of consent attachments accommodates the complexity of modern privacy requirements. Rather than simple binary consent decisions, the attachments support granular permissions that can vary by purpose, time period, and processing activity. For example, a customer might consent to conversation recording for quality assurance purposes but decline consent for AI training applications. The consent attachment captures these nuanced decisions and enables automated systems to respect the specified limitations.

    Temporal management within consent attachments addresses the dynamic nature of privacy consent. The attachments include expiration timestamps that specify when consent expires and requires renewal. They also support indefinite consent with periodic revalidation requirements, accommodating different regulatory frameworks and organizational policies. When consent expires or is revoked, systems processing the vCon can immediately detect the status change and respond appropriately.

    The cryptographic foundations of consent attachments ensure their authenticity and integrity. Digital signatures verify that consent decisions came from authorized parties and haven't been tampered with during data processing or transmission. Hash functions and integrity checks prevent unauthorized modifications to consent decisions, ensuring that privacy protections cannot be circumvented through technical manipulation.

    Integration with emerging privacy standards further enhances the utility of consent attachments. The framework supports the AI Preferences vocabulary developed by the Internet Engineering Task Force, enabling standardized expression of consent for artificial intelligence and machine learning applications. This standardization is particularly important as AI governance regulations emerge worldwide, requiring organizations to demonstrate explicit consent for AI training and automated decision-making systems.

    Consent attachments also support multiple proof mechanisms that accommodate different organizational and regulatory requirements. Cryptographic proofs provide mathematical certainty for high-security environments, while documented consent processes support traditional compliance frameworks. The attachments can reference external consent forms, embed digital signatures, or document verbal consent given during conversations, providing flexibility while maintaining verification capabilities.

    The practical implementation of consent attachments transforms how organizations approach privacy compliance. Instead of requiring separate consent verification processes before each data use, systems can automatically check consent status by examining the embedded attachments. This automation reduces compliance burden while improving privacy protection, as consent verification becomes a natural part of data processing workflows rather than an additional overhead.

    Privacy Rights and Automated Compliance

    The automation of privacy rights fulfillment represents one of the most significant practical benefits of integrating vCons with SCITT transparency services. Traditional approaches to privacy rights management rely heavily on manual processes that are slow, error-prone, and difficult to scale. The integrated framework enables automated responses to privacy requests while maintaining the accuracy and verifiability required for regulatory compliance.

    When individuals exercise their right to access personal data, traditional systems require manual searches across multiple databases, applications, and archive systems. This process is time-consuming, expensive, and often incomplete, as organizations struggle to identify all systems that might contain relevant data. The vCon framework fundamentally changes this dynamic by creating comprehensive conversation records that include complete audit trails of data processing activities.

    The SCITT transparency service acts as a central index of all conversation-related activities, enabling automated identification of relevant data in response to access requests. When someone requests information about how their conversation data was processed, the system can query the transparency service to identify all vCons containing their information, trace all processing activities performed on that data, and generate comprehensive reports without manual intervention.

    Data portability rights, which require organizations to provide personal data in machine-readable formats, are naturally supported by the vCon framework. Since vCons use standardized JSON formats with well-defined schemas, individuals can receive their conversation data in formats that are both human-readable and compatible with other systems. This eliminates the common practice of providing data exports in proprietary formats that are difficult to use or transfer to other services.

    The right to rectification, which allows individuals to correct inaccurate personal data, benefits from the immutable audit trails provided by SCITT. When corrections are made to conversation data, the changes are documented in the transparency service, creating a verifiable record of what information was changed, when the change occurred, and who authorized the modification. This transparency builds trust while ensuring that corrections cannot be disputed later.

    Perhaps most complex is the automation of data deletion rights, commonly known as the "right to be forgotten." Traditional deletion processes struggle with distributed data architectures where copies of personal information might exist across multiple systems, backup archives, and third-party processors. The vCon framework addresses this challenge through comprehensive tracking of data distribution combined with automated deletion coordination.

    When consent is revoked or deletion is requested, the SCITT transparency service identifies all systems that have received copies of the relevant vCons. Automated deletion requests are then sent to these systems, with responses tracked and verified through additional SCITT entries. This process ensures that deletion requests are honored comprehensively rather than only in the primary system where the request was received.

    The framework also supports partial deletion scenarios where individuals might revoke consent for specific purposes while maintaining consent for others. For example, someone might withdraw consent for AI training while maintaining consent for conversation recording for quality assurance. The system can automatically redact or modify vCons to reflect these granular consent changes while maintaining data integrity for authorized uses.

    Compliance reporting becomes significantly more efficient through automated privacy rights management. Organizations can generate real-time reports on privacy request volumes, response times, and completion rates using data automatically collected through the SCITT transparency service. These reports provide the metrics and documentation required for regulatory reporting while reducing the manual effort traditionally required for compliance reporting.

    The verification capabilities provided by SCITT receipts transform the relationship between organizations and privacy regulators. Instead of requiring extensive document production during privacy audits, organizations can provide cryptographic proof of their privacy rights fulfillment processes. Regulators can verify compliance through mathematical certainty rather than document review, enabling more efficient oversight while providing stronger privacy protections.

    Business Benefits and Competitive Advantages

    Organizations implementing vCon lifecycle management with SCITT transparency services realize significant business benefits that extend far beyond regulatory compliance. These advantages touch multiple aspects of business operations, from operational efficiency and risk management to customer trust and competitive positioning.

    Operational efficiency improvements emerge from the standardization and automation enabled by the vCon framework. Currently, most organizations manage conversation data through fragmented systems with incompatible formats, manual processes, and disconnected privacy controls. The integration of vCons with SCITT creates unified workflows that reduce manual effort, eliminate data format conversion overhead, and enable automated compliance processes.

    Customer service organizations particularly benefit from standardized conversation management. Representatives can access complete conversation histories regardless of the original communication channel, enabling more effective customer interactions. The embedded consent information ensures that representatives understand exactly what data uses are authorized, preventing privacy violations while enabling personalized service.

    Risk management benefits accrue from the comprehensive audit trails and tamper-evident records provided by SCITT transparency services. Organizations can demonstrate compliance with privacy regulations through cryptographic proof rather than documentary evidence, reducing regulatory risk and potential penalties. The immutable nature of SCITT records provides legal protections in disputes about consent decisions or data processing activities.

    The transparency provided by this framework enables new forms of customer engagement based on trust and data ownership. Organizations can provide customers with real-time visibility into how their conversation data is being used, what insights are being generated, and what value is being created. This transparency can transform privacy from a compliance burden into a competitive differentiator.

    Cost reduction occurs across multiple dimensions of privacy management. Automated privacy rights fulfillment reduces the labor costs associated with manual data searches and report generation. Standardized data formats eliminate the need for custom integration work when implementing new conversation analysis tools or communication platforms. The prevention of privacy violations through embedded consent controls reduces potential regulatory penalties and reputation damage.

    Innovation enablement represents a particularly significant benefit for organizations developing artificial intelligence and machine learning capabilities. The granular consent management provided by consent attachments enables organizations to use conversation data for AI training while respecting individual privacy preferences. This capability becomes increasingly important as AI governance regulations emerge worldwide, requiring explicit consent for AI applications.

    Market differentiation through privacy leadership provides competitive advantages in markets where consumers are increasingly privacy-conscious. Organizations that can demonstrate transparent, respectful data practices through verifiable audit trails gain customer trust and loyalty. This differentiation is particularly valuable in industries where data processing is central to service delivery, such as healthcare, financial services, and telecommunications.

    The global nature of the framework provides advantages for multinational organizations that must comply with diverse privacy regulations across different jurisdictions. Rather than implementing separate compliance systems for each regulatory framework, organizations can use the vCon framework to meet the requirements of multiple regulations through a single, standardized approach.

    Vendor independence emerges from the standardized nature of vCons, reducing reliance on proprietary data formats and enabling organizations to choose best-of-breed solutions for different aspects of conversation management. This flexibility supports innovation and cost optimization while reducing vendor lock-in risks.

    Partnership and collaboration opportunities expand when organizations can share conversation data with verified consent and audit trails. Research collaborations, industry benchmarking, and supply chain partnerships become more feasible when all parties can verify that data sharing complies with privacy requirements and consent limitations.

    Implementation Considerations and Best Practices

    Successful implementation of vCon lifecycle management with SCITT transparency services requires careful planning, staged deployment, and attention to both technical and organizational change management factors. Organizations embarking on this transformation should consider several key dimensions of implementation to maximize benefits while minimizing risks and disruption.

    Technical architecture planning forms the foundation of successful implementation. Organizations must assess their current conversation data landscape, identifying all systems that capture, process, store, or analyze conversational information. This assessment typically reveals significant complexity, with conversation data scattered across customer relationship management systems, telephony platforms, email servers, chat applications, video conferencing tools, and various analytics platforms.

    The migration strategy should prioritize high-value, high-risk conversation types for initial implementation. Customer service interactions, sales calls, and other conversations involving sensitive personal information typically provide the greatest immediate benefits from enhanced privacy protection and compliance automation. These conversations also face the highest regulatory scrutiny, making the compliance benefits particularly valuable.

    Consent collection process redesign represents a critical implementation consideration. Organizations must evaluate their current consent mechanisms and design new processes that integrate seamlessly with vCon creation workflows. This often involves updating communication platform configurations, training customer service representatives, and implementing automated consent detection for different conversation types.

    Staff training and change management requirements extend beyond technical implementation to encompass new operational procedures, privacy awareness, and compliance workflows. Customer service representatives need training on consent verification procedures and privacy limitation awareness. IT staff require education on vCon processing workflows and SCITT verification procedures. Legal and compliance teams need familiarity with new audit trail capabilities and automated reporting functions.

    Integration with existing systems requires careful API design and data migration planning. Organizations typically cannot replace all conversation management systems simultaneously, necessitating hybrid architectures where vCon-enabled systems interoperate with legacy platforms. This requires robust transformation and synchronization mechanisms that maintain data integrity and consent consistency across system boundaries.

    Security considerations encompass both data protection and access control requirements. vCons containing sensitive conversation data require encryption in transit and at rest, with careful key management and access control implementation. SCITT transparency services require secure authentication and authorization mechanisms that prevent unauthorized statement submission while maintaining the transparency benefits of the framework.

    Performance and scalability planning must account for the potentially massive volumes of conversation data in enterprise environments. Large organizations might process millions of conversations daily, requiring SCITT transparency services and vCon processing systems that can handle high transaction volumes with acceptable latency and reliability.

    Vendor selection and partnership decisions significantly impact implementation success. Organizations must evaluate SCITT transparency service providers, vCon processing platforms, and consent management solutions based on technical capabilities, compliance certifications, and long-term viability. The standardized nature of vCons provides flexibility in vendor selection while requiring due diligence on implementation quality and interoperability.

    Monitoring and measurement frameworks should be established to track implementation progress and business benefits realization. Key metrics might include privacy rights request processing times, consent verification automation rates, audit trail completeness, and customer satisfaction with transparency features. These measurements enable continuous improvement and demonstrate return on investment for privacy technology initiatives.

    Regulatory engagement and compliance validation represent ongoing requirements rather than one-time implementation tasks. Organizations should engage with relevant privacy regulators to understand how vCon-based compliance demonstrations will be evaluated and what documentation or verification procedures are expected. This proactive engagement can inform implementation decisions and reduce regulatory uncertainty.

    Future Implications and Industry Transformation

    The widespread adoption of vCon lifecycle management with SCITT transparency services has the potential to transform entire industries and reshape how society approaches privacy protection and data governance. These changes extend far beyond individual organizations to encompass market dynamics, regulatory frameworks, and consumer expectations.

    Industry standardization around vCons could eliminate many of the interoperability challenges that currently plague conversation management systems. When all communication platforms, analytics tools, and customer relationship management systems support standardized vCon formats, organizations gain unprecedented flexibility in system selection and vendor management. This standardization could accelerate innovation by reducing integration overhead and enabling specialized solutions to focus on their core value propositions.

    The transparency provided by SCITT-based audit trails could fundamentally change the relationship between organizations and privacy regulators. Rather than periodic audits based on document review, regulators could implement continuous monitoring systems that verify compliance through real-time cryptographic proof. This shift could reduce regulatory burden for compliant organizations while enabling more effective oversight of privacy violations.

    Consumer expectations around data transparency and control will likely evolve as these capabilities become more widely available. Individuals may begin to expect real-time visibility into how their conversation data is being used, similar to how financial services now provide real-time transaction notifications. Organizations that can provide this transparency will gain competitive advantages, while those that cannot may face customer attrition.

    Artificial intelligence governance could be revolutionized through granular consent management capabilities. As governments worldwide develop AI regulations requiring explicit consent for training data use, the ability to track and verify consent for AI applications becomes a critical competitive capability. Organizations with robust consent management frameworks will be better positioned to develop AI capabilities within regulatory constraints.

    Cross-border data governance could be simplified through standardized privacy frameworks that work across jurisdictions. Rather than implementing separate compliance systems for each country's privacy regulations, multinational organizations could use vCon-based frameworks that meet the requirements of multiple regulatory systems simultaneously. This could reduce compliance costs while improving privacy protection consistency.

    New business models could emerge around privacy transparency and data governance services. Organizations might offer premium services that provide enhanced privacy transparency, specialized consent management for sensitive industries, or privacy audit services based on SCITT verification capabilities. These new service categories could create economic incentives for privacy protection beyond regulatory compliance.

    The democratization of privacy technology could extend advanced privacy capabilities to smaller organizations that currently lack the resources for sophisticated privacy management systems. Cloud-based vCon processing and SCITT transparency services could make enterprise-grade privacy capabilities accessible to organizations of all sizes, raising the overall level of privacy protection across the economy.

    Academic research and policy development could benefit from standardized conversation data formats that enable privacy-preserving analysis of communication patterns and privacy behavior. Researchers could study the effectiveness of different consent mechanisms, analyze the impact of privacy regulations, and develop improved privacy protection technologies using anonymized vCon datasets.

    International cooperation on privacy protection could be enhanced through standardized frameworks that enable cross-border verification of privacy compliance. Trade agreements, mutual recognition arrangements, and international privacy frameworks could incorporate vCon-based verification mechanisms that reduce friction while maintaining protection standards.

    The evolution toward privacy-first data architectures represents a fundamental shift in how technology systems are designed and operated. Rather than treating privacy as an add-on feature, future systems could embed privacy protection as a core architectural principle, with consent management, audit trails, and transparency features built into the foundation of data processing systems.

    Conclusion

    The integration of Virtualized Conversations with Supply Chain Integrity, Transparency, and Trust protocols represents more than a technological advancement; it embodies a fundamental reimagining of how organizations can balance the tremendous value of conversational data with the imperative of privacy protection and regulatory compliance. This framework transforms privacy from a constraint on business operations into an enabler of trust, innovation, and competitive advantage.

    The challenges that this framework addresses are not merely technical but reflect deeper societal questions about data ownership, consent, and the balance between individual privacy rights and collective benefits from data processing. By embedding privacy protection directly into data containers and creating immutable audit trails of data processing activities, the vCon-SCITT integration provides a foundation for rebuilding trust between organizations and the individuals whose data they process.

    The business case for adoption extends far beyond regulatory compliance to encompass operational efficiency, risk management, customer trust, and innovation enablement. Organizations that embrace this framework position themselves to thrive in an increasingly privacy-conscious marketplace while gaining the flexibility to develop new services and capabilities within a foundation of verifiable privacy protection.

    Perhaps most significantly, this approach provides a pathway for responsible innovation in artificial intelligence and machine learning applications. As societies worldwide grapple with the governance of AI systems, the ability to demonstrate explicit, granular consent for training data use becomes essential for maintaining public trust and regulatory approval for AI development.

    The transformation potential extends beyond individual organizations to encompass entire industries and regulatory frameworks. Standardized approaches to conversation data management and privacy protection could reduce compliance costs, enable new forms of collaboration, and provide regulators with more effective oversight mechanisms based on cryptographic verification rather than document review.

    The future envisioned by this framework is one where privacy protection enhances rather than constrains business value creation. Organizations that can demonstrate transparent, respectful data practices through verifiable audit trails will earn customer trust and loyalty while gaining access to rich conversational data for legitimate business purposes. This alignment of privacy protection with business success creates sustainable incentives for responsible data management that extend far beyond regulatory compliance requirements.

    As this framework matures and gains adoption, it has the potential to serve as a model for privacy protection in other domains beyond conversational data. The principles of embedded consent management, immutable audit trails, and automated compliance could be applied to healthcare data, financial information, location data, and other categories of personal information that require sophisticated privacy protection.

    The path forward requires collaboration between technology vendors, privacy advocates, regulators, and organizations across industries to refine standards, develop best practices, and create the ecosystem of tools and services needed for widespread adoption. This collaboration must balance innovation enablement with robust privacy protection, ensuring that technological advancement serves human values and societal benefit.

    Ultimately, the vCon lifecycle management framework with SCITT transparency services represents a vision of a future where privacy protection and business innovation are not opposing forces but complementary capabilities that together create more trustworthy, more valuable, and more sustainable approaches to data processing in our increasingly connected world.

    Can stop processing chain

    Failures don't stop main processing

    Archive Integration: Stores audit records in external archive systems

    Performance Profiling: Detailed performance analysis

    Aspect

    Links

    Tracers

    Purpose

    Transform/process vCon data

    Observe/monitor data flow

    Data Modification

    Can modify vCon content

    Never modify vCon content

    Execution Timing

    Sequential in processing chain

    Execute before/after each link

    Return Value

    Return vCon UUID for chaining

    Return boolean success status

    Failure Impact

    Base Adapter Pattern

    Use this as the foundation for all adapters:

    Key Patterns to Follow

    1. vCon Creation

    Always start with:

    2. Party Processing

    Create a mapping between source participants and vCon parties:

    3. Dialog Processing

    Handle different dialog types:

    4. Timestamp Handling

    Always convert timestamps to ISO format:

    Media Handling Patterns

    Audio/Video Content

    Handle media files properly:

    Transfer Dialogs

    Handle call transfers:

    Incomplete Dialogs

    Handle failed conversations:

    Error Handling Requirements

    Robust Data Extraction

    Always handle missing or malformed data:

    Validation and Fallbacks

    Provide fallbacks for missing required data:

    Common Adapter Templates

    Chat System Adapter

    Call Center Adapter

    Critical Requirements

    1. Always Validate

    2. Handle All Dialog Types

    Support these dialog types based on source data:

    • "text" - Text messages, chat, transcripts

    • "recording" - Audio recordings

    • "video" - Video calls/recordings

    • "transfer" - Call transfers

    • "incomplete" - Failed/incomplete calls

    3. Use Proper MIME Types

    Use these MIME types:

    • Text: "text/plain"

    • Audio: "audio/wav", "audio/mp3", "audio/x-wav", "audio/x-mp3"

    • Video: "video/mp4", "video/webm", "video/x-mp4"

    • Email: "message/rfc822"

    4. Include Extensions and Must-Support (vCon 0.3.0)

    Testing Pattern

    Always include this test structure:

    Key Considerations for LLMs

    1. Always use the base adapter pattern - don't create adapters from scratch

    2. Handle missing data gracefully - provide defaults and fallbacks

    3. Validate all timestamps - convert to ISO 8601 format

    4. Map participant IDs correctly - maintain consistent party references

    5. Include proper error handling - log errors and provide meaningful messages

    6. Use appropriate dialog types - match the source content type

    7. Add relevant metadata - use tags and extensions appropriately

    8. Test the generated vCon - always validate before returning

    When generating adapter code, focus on the specific source system requirements while following these patterns and ensuring compliance with the vCon specification.

    # Simplified execution flow
    for link_index, link_name in enumerate(links):
        if link_index == 0:
            # Execute tracers before first link
            _process_tracers(vcon_id, vcon_id, links, -1)
        
        # Execute the processing link
        result = execute_link(link_name, vcon_id)
        
        # Execute tracers after link completion
        _process_tracers(result, vcon_id, links, link_index)
    
    tracers:
      jlinc:
        module: tracers.jlinc
        options:
          data_store_api_url: http://jlinc-server:9090
          data_store_api_key: your_data_store_api_key
          archive_api_url: http://jlinc-server:9090
          archive_api_key: your_archive_api_key
          system_prefix: VCONTest
          agreement_id: 00000000-0000-0000-0000-000000000000
          hash_event_data: True
          dlq_vcon_on_error: True
    tracers:
      tracer_name:
        module: tracers.module_name
        options:
          # Tracer-specific configuration options
    tracers:
      jlinc_audit:
        module: tracers.jlinc
        options:
          # JLINC configuration
          
      compliance_logger:
        module: tracers.compliance
        options:
          # Compliance logging configuration
          
      metrics_collector:
        module: tracers.metrics
        options:
    def run(
        in_vcon_uuid: str,      # Input vCon UUID
        out_vcon_uuid: str,     # Output vCon UUID  
        tracer_name: str,       # Name of this tracer instance
        links: list[str],       # List of all links in the chain
        link_index: int,        # Current link index (-1 for pre-chain)
        opts: dict = {}         # Tracer configuration options
    ) -> bool:
        """
        Execute tracer logic for a vCon processing step.
        
        Args:
            in_vcon_uuid: UUID of vCon entering the processing step
            out_vcon_uuid: UUID of vCon exiting the processing step
            tracer_name: Name of this tracer instance from config
            links: Complete list of links in the processing chain
            link_index: Index of current link (-1 for pre-chain execution)
            opts: Tracer-specific configuration options
            
        Returns:
            bool: True if tracer executed successfully, False otherwise
        """
    from lib.logging_utils import init_logger
    from lib.vcon_redis import VconRedis
    
    logger = init_logger(__name__)
    
    default_options = {
        "api_url": "http://example.com/api",
        "api_key": "",
        "enabled": True
    }
    
    def run(in_vcon_uuid, out_vcon_uuid, tracer_name, links, link_index, opts=default_options):
        """Example tracer implementation"""
        
        if not opts.get("enabled", True):
            logger.debug(f"Tracer {tracer_name} is disabled")
            return True
        
        try:
            # Get vCon data
            vcon_redis = VconRedis()
            vcon_obj = vcon_redis.get_vcon(out_vcon_uuid)
            
            if not vcon_obj:
                logger.error(f"Could not retrieve vCon {out_vcon_uuid}")
                return False
            
            # Process tracer logic
            logger.info(f"Executing {tracer_name} tracer for vCon {out_vcon_uuid}")
            
            # Your tracer logic here
            # - Send data to external systems
            # - Create audit records
            # - Generate metrics
            # - Log compliance information
            
            return True
            
        except Exception as e:
            logger.error(f"Tracer {tracer_name} failed: {e}")
            return False
            
            
    logger.info(
        "Completed tracer %s (module: %s) for vCon: %s in %s seconds",
        tracer_name,
        tracer_module_name,
        out_vcon_uuid,
        tracer_processing_time,
        extra={
            "tracer_processing_time": tracer_processing_time,
            "tracer_name": tracer_name,
            "tracer_module_name": tracer_module_name
        }
    )
    from abc import ABC, abstractmethod
    from typing import Dict, List, Any, Optional, Union
    from vcon import Vcon, Party, Dialog
    from datetime import datetime, timezone
    import json
    import base64
    import logging
    class BaseVconAdapter(ABC):
        """Base class for all vCon adapters."""
        
        def __init__(self, config: Dict[str, Any]):
            self.config = config
            self.validation_errors = []
            self.logger = logging.getLogger(self.__class__.__name__)
        
        @abstractmethod
        def extract_data(self, source: Any) -> Dict[str, Any]:
            """Extract raw data from the source system."""
            pass
        
        @abstractmethod
        def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
            """Transform raw data into a vCon object."""
            pass
        
        def validate_vcon(self, vcon: Vcon) -> bool:
            """Validate the generated vCon."""
            is_valid, errors = vcon.is_valid()
            self.validation_errors = errors
            return is_valid
        
        def process(self, source: Any) -> Vcon:
            """Main processing pipeline."""
            raw_data = self.extract_data(source)
            vcon = self.transform_to_vcon(raw_data)
            
            if not self.validate_vcon(vcon):
                raise ValueError(f"Invalid vCon generated: {self.validation_errors}")
            
            return vcon
    def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
        vcon = Vcon.build_new()
        
        # Add metadata tags
        vcon.add_tag("source", "your_system_name")
        vcon.add_tag("adapter_version", "1.0")
        
        # Process data...
        return vcon
    # Build participant mapping
    participant_map = {}
    for i, participant in enumerate(raw_data.get("participants", [])):
        party = Party(
            name=participant.get("name"),
            tel=participant.get("phone"),
            mailto=participant.get("email"),
            role=participant.get("role", "participant"),
            # Add new vCon 0.3.0 fields if available
            sip=participant.get("sip_uri"),
            did=participant.get("decentralized_id"),
            timezone=participant.get("timezone")
        )
        vcon.add_party(party)
        participant_map[participant["id"]] = i
    for message in raw_data.get("messages", []):
        dialog = Dialog(
            type="text",  # or "recording", "video", "transfer", "incomplete"
            start=self.parse_timestamp(message["timestamp"]),
            parties=[participant_map[message["sender_id"]]],
            originator=participant_map[message["sender_id"]],
            mimetype="text/plain",
            body=message["content"],
            # Add new vCon 0.3.0 fields
            session_id=message.get("session_id"),
            application=message.get("app_name"),
            message_id=message.get("message_id")
        )
        vcon.add_dialog(dialog)
    def parse_timestamp(self, timestamp) -> str:
        """Convert various timestamp formats to ISO 8601."""
        if isinstance(timestamp, datetime):
            return timestamp.isoformat()
        elif isinstance(timestamp, str):
            try:
                # Try parsing common formats
                from dateutil import parser
                return parser.parse(timestamp).isoformat()
            except:
                return datetime.now(timezone.utc).isoformat()
        elif isinstance(timestamp, (int, float)):
            # Assume Unix timestamp
            return datetime.fromtimestamp(timestamp, timezone.utc).isoformat()
        else:
            return datetime.now(timezone.utc).isoformat()
    def add_media_dialog(self, media_data: Dict[str, Any], parties: List[int]) -> Dialog:
        """Add audio or video dialog."""
        if media_data.get("url"):
            # External media reference
            dialog = Dialog(
                type="recording" if media_data["type"] == "audio" else "video",
                start=self.parse_timestamp(media_data["timestamp"]),
                parties=parties,
                mimetype=media_data.get("mimetype", "audio/wav"),
                url=media_data["url"],
                duration=media_data.get("duration"),
                content_hash=media_data.get("hash")  # New in vCon 0.3.0
            )
        else:
            # Inline media (base64 encoded)
            dialog = Dialog(
                type="recording" if media_data["type"] == "audio" else "video",
                start=self.parse_timestamp(media_data["timestamp"]),
                parties=parties,
                mimetype=media_data.get("mimetype", "audio/wav"),
                body=media_data["base64_content"],
                encoding="base64",
                filename=media_data.get("filename")
            )
        
        return dialog
    def add_transfer_dialog(self, transfer_data: Dict[str, Any]) -> None:
        """Add transfer dialog for call transfers."""
        vcon.add_transfer_dialog(
            start=self.parse_timestamp(transfer_data["timestamp"]),
            transfer_data={
                "reason": transfer_data.get("reason", "Call transferred"),
                "from": transfer_data.get("from_number"),
                "to": transfer_data.get("to_number"),
                "transfer_target": transfer_data.get("target_party_index"),
                "transferor": transfer_data.get("transferor_party_index"),
                "transferee": transfer_data.get("transferee_party_index")
            },
            parties=transfer_data.get("involved_parties", [])
        )
    def add_incomplete_dialog(self, failed_call: Dict[str, Any]) -> None:
        """Add incomplete dialog for failed calls."""
        # Map common failure reasons to vCon dispositions
        disposition_map = {
            "no_answer": "no-answer",
            "busy": "busy",
            "failed": "failed",
            "hung_up": "hung-up",
            "voicemail": "voicemail-no-message",
            "congestion": "congestion"
        }
        
        disposition = disposition_map.get(
            failed_call.get("reason", "failed").lower(), 
            "failed"
        )
        
        vcon.add_incomplete_dialog(
            start=self.parse_timestamp(failed_call["timestamp"]),
            disposition=disposition,
            details=failed_call.get("details", {}),
            parties=failed_call.get("involved_parties", [])
        )
    def extract_data(self, source: Any) -> Dict[str, Any]:
        """Extract data with error handling."""
        try:
            if isinstance(source, str):
                # File path
                with open(source, 'r') as f:
                    return json.load(f)
            elif isinstance(source, dict):
                # Direct data
                return source
            else:
                raise ValueError(f"Unsupported source type: {type(source)}")
        except Exception as e:
            self.logger.error(f"Failed to extract data: {e}")
            raise
    def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
        vcon = Vcon.build_new()
        
        # Ensure required fields exist
        if not raw_data.get("participants"):
            # Create a default participant if none exist
            default_party = Party(name="Unknown", role="participant")
            vcon.add_party(default_party)
            participant_map = {"default": 0}
        else:
            participant_map = self.process_participants(raw_data, vcon)
        
        # Handle missing timestamps
        default_timestamp = datetime.now(timezone.utc).isoformat()
        
        # Process messages with fallbacks
        for message in raw_data.get("messages", []):
            dialog = Dialog(
                type="text",
                start=self.parse_timestamp(message.get("timestamp", default_timestamp)),
                parties=[participant_map.get(message.get("sender_id"), 0)],
                originator=participant_map.get(message.get("sender_id"), 0),
                mimetype="text/plain",
                body=message.get("content", "")
            )
            vcon.add_dialog(dialog)
        
        return vcon
    class ChatSystemAdapter(BaseVconAdapter):
        """Template for chat/messaging systems."""
        
        def extract_data(self, chat_file: str) -> Dict[str, Any]:
            with open(chat_file, 'r') as f:
                return json.load(f)
        
        def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
            vcon = Vcon.build_new()
            vcon.add_tag("source", "chat_system")
            
            # Process participants
            participant_map = {}
            for i, user in enumerate(raw_data.get("users", [])):
                party = Party(
                    name=user.get("display_name"),
                    mailto=user.get("email"),
                    role="participant"
                )
                vcon.add_party(party)
                participant_map[user["id"]] = i
            
            # Process messages
            for msg in raw_data.get("messages", []):
                dialog = Dialog(
                    type="text",
                    start=self.parse_timestamp(msg["timestamp"]),
                    parties=[participant_map[msg["user_id"]]],
                    originator=participant_map[msg["user_id"]],
                    mimetype="text/plain",
                    body=msg["text"]
                )
                vcon.add_dialog(dialog)
            
            return vcon
    class CallCenterAdapter(BaseVconAdapter):
        """Template for call center systems."""
        
        def extract_data(self, call_record: Dict[str, Any]) -> Dict[str, Any]:
            return call_record
        
        def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
            vcon = Vcon.build_new()
            vcon.add_tag("source", "call_center")
            vcon.add_tag("call_id", raw_data.get("call_id"))
            
            # Add caller
            caller = Party(
                tel=raw_data.get("caller_number"),
                name=raw_data.get("caller_name"),
                role="caller"
            )
            vcon.add_party(caller)
            
            # Add agent
            agent = Party(
                tel=raw_data.get("agent_extension"),
                name=raw_data.get("agent_name"),
                role="agent"
            )
            vcon.add_party(agent)
            
            # Add call recording if available
            if raw_data.get("recording_url"):
                recording_dialog = Dialog(
                    type="recording",
                    start=self.parse_timestamp(raw_data["start_time"]),
                    parties=[0, 1],
                    mimetype="audio/wav",
                    url=raw_data["recording_url"],
                    duration=raw_data.get("duration")
                )
                vcon.add_dialog(recording_dialog)
            
            # Add transcript if available
            if raw_data.get("transcript"):
                for entry in raw_data["transcript"]:
                    dialog = Dialog(
                        type="text",
                        start=self.parse_timestamp(entry["timestamp"]),
                        parties=[entry.get("speaker_id", 0)],
                        originator=entry.get("speaker_id", 0),
                        mimetype="text/plain",
                        body=entry["text"]
                    )
                    vcon.add_dialog(dialog)
            
            return vcon
    # At the end of transform_to_vcon
    is_valid, errors = vcon.is_valid()
    if not is_valid:
        self.logger.error(f"Generated invalid vCon: {errors}")
        # Either fix the issues or raise an exception
    # Add extensions if using advanced features
    if using_video:
        vcon.add_extension("video")
    if using_encryption:
        vcon.add_extension("encryption")
        vcon.add_must_support("encryption")
    def test_adapter():
        """Test the adapter with sample data."""
        adapter = YourAdapter({})
        
        sample_data = {
            # Your test data structure
        }
        
        vcon = adapter.process(sample_data)
        is_valid, errors = vcon.is_valid()
        
        assert is_valid, f"Invalid vCon: {errors}"
        assert len(vcon.parties) > 0, "No parties found"
        assert len(vcon.dialog) > 0, "No dialog found"
        
        return vcon

    Privacy-First Conversation Management

    A Technical Whitepaper on Standardized Consent in Virtualized Conversations

    Executive Summary

    In our increasingly digital world, voice conversations generate valuable data for businesses while creating significant privacy obligations. The challenge isn't just collecting consent—it's managing that consent throughout the entire data lifecycle while ensuring compliance with evolving privacy regulations like GDPR, CCPA, and emerging AI governance frameworks.

    The vCon (Virtualized Conversation) consent attachment specification solves this challenge by embedding structured consent information directly within conversation containers. This approach ensures that consent travels with the data, enabling automated compliance checking, granular permission management, and transparent audit trails—all while supporting modern AI applications that depend on conversational data.

    Key Benefits:

    Privacy by Design: Consent is embedded directly in conversation data containers

  • Regulatory Compliance: Built-in support for GDPR, CCPA, HIPAA, and AI governance frameworks

  • Granular Control: Purpose-specific permissions (recording, transcription, AI training, etc.)

  • Cryptographic Verification: Tamper-evident consent records with blockchain-style transparency

  • Automated Processing: Machine-readable consent enables real-time compliance decisions

  • The Problem: Consent Management in the AI Era

    Current Challenges

    Traditional consent management systems face several critical limitations:

    1. Disconnected Consent: Consent records are stored separately from the data they govern, creating compliance gaps

    2. Static Permissions: Binary consent models can't handle nuanced use cases like "yes to transcription, no to AI training"

    3. Audit Complexity: Proving compliance requires reconstructing consent decisions across multiple systems

    4. AI Blindness: Existing systems weren't designed for AI-specific use cases like model training and inference

    Real-World Example: Contact Center Compliance

    Consider a healthcare contact center handling patient calls:

    • Initial Recording: Patient consents to call recording for quality assurance

    • Transcription: Automated speech-to-text for record keeping

    • AI Analysis: Sentiment analysis to improve service quality

    • Model Training: Using anonymized conversations to train better AI assistants

    Each step requires different permissions under different regulations (HIPAA for healthcare data, GDPR for EU residents, state privacy laws for California residents). Traditional systems struggle to track these granular permissions as data moves through various processing stages.

    The vCon Solution: Structured Conversation Containers

    What is a vCon?

    A vCon (Virtualized Conversation) is a standardized JSON container that packages all information related to a conversation:

    • Parties: Who participated in the conversation

    • Dialog: The actual conversation content (audio, text, video)

    • Analysis: Processed insights (transcripts, sentiment, summaries)

    • Attachments: Supporting documents and metadata

    The Consent Attachment: Privacy-Embedded Data

    The consent attachment extends vCons with structured consent information that travels with the conversation data. This ensures that privacy preferences are never separated from the data they govern.

    Core Structure

    Key Features and Benefits

    1. Granular Purpose-Based Permissions

    Unlike binary consent models, vCon consent attachments support granular permissions for specific purposes:

    Standard Purposes:

    • recording - Permission to record the conversation

    • transcription - Permission to convert speech to text

    • analysis - Permission for sentiment analysis, summarization

    • storage - Permission for long-term data retention

    • sharing - Permission to share with third parties

    AI-Specific Purposes (AI Preferences Vocabulary):

    • ai - General AI training and model development

    • genai - Generative AI training (for content creation models)

    • tdm - Text and Data Mining for analytical purposes

    • inference - Using data as input to trained AI models

    2. Cryptographic Verification and Transparency

    The specification integrates with SCITT (Supply Chain Integrity, Transparency, and Trust) to provide:

    • Tamper-evident records: Cryptographic signatures prevent consent modification

    • Transparency ledgers: Immutable audit trails for compliance verification

    • Receipt mechanisms: Cryptographic proof of consent registration

    • Revocation tracking: Transparent consent withdrawal processes

    3. Temporal Management

    Consent attachments include sophisticated time-based controls:

    • Expiration timestamps: Automatic consent invalidation

    • Status intervals: Configurable verification frequency based on data sensitivity

    • Clock skew tolerance: Handling time differences across distributed systems

    • Indefinite consent: Support for open-ended permissions with periodic revalidation

    4. Regulatory Compliance Integration

    The specification directly addresses requirements from major privacy frameworks:

    GDPR Compliance:

    • Right of access to consent records

    • Right of rectification for consent information

    • Right to be forgotten through consent revocation

    • Right of portability for consent data export

    CCPA Support:

    • Consumer right to know about data processing

    • Right to delete personal information

    • Right to opt-out of data sales

    • Non-discrimination for privacy choices

    HIPAA Integration:

    • Healthcare-specific consent management

    • Business associate agreement compliance

    • Audit trail requirements

    • Breach notification support

    Technical Implementation

    Architecture Overview

    The vCon consent ecosystem consists of several integrated components:

    1. vCon Containers: JSON-based conversation packages with embedded consent

    2. Consent Ledger: SCITT-based transparency service for consent state management

    3. Processing Engines: Applications that respect consent decisions during data processing

    4. Verification Services: Tools for validating consent status and cryptographic proofs

    Integration Patterns

    Pattern 1: Real-time Consent Verification

    Pattern 2: Consent Ledger Integration

    Security Considerations

    The specification addresses security through multiple mechanisms:

    Cryptographic Protection:

    • COSE (CBOR Object Signing and Encryption) for consent signatures

    • Certificate chain validation for signing authority

    • Content integrity verification for external references

    Access Control:

    • Minimal privilege consent verification

    • Role-based access to consent data

    • Audit logging for all consent operations

    Network Security:

    • TLS 1.2+ for all consent ledger communications

    • Certificate pinning for critical services

    • Rate limiting to prevent abuse

    Use Cases and Applications

    Use Case 1: Multi-Modal Customer Service

    Scenario: A telecommunications company handles customer inquiries across voice, chat, and email channels.

    Implementation:

    • Each interaction creates a vCon with appropriate consent attachments

    • Consent travels with data as conversations escalate between channels

    • AI analysis respects granular permissions across all touchpoints

    • Compliance reporting aggregates consent decisions automatically

    Benefits:

    • Unified privacy management across communication channels

    • Automated compliance with varying regulatory requirements

    • Enhanced customer trust through transparent consent handling

    Use Case 2: Healthcare Telemedicine Platform

    Scenario: A telemedicine platform conducts video consultations with patients globally.

    Implementation:

    • Patient consent captured before session recording begins

    • Consent attachments specify purposes: medical records, quality assurance, research

    • Integration with Electronic Health Record (EHR) systems

    • Automatic deletion of recordings when consent expires

    Benefits:

    • HIPAA compliance for US patients

    • GDPR compliance for EU patients

    • Audit trails for regulatory inspections

    • Patient control over data usage

    Use Case 3: AI Training Data Pipeline

    Scenario: A technology company develops conversational AI using customer service recordings.

    Implementation:

    • Historical conversations tagged with retroactive consent analysis

    • New conversations include AI-specific consent categories

    • Data pipeline automatically filters based on AI Preferences vocabulary

    • Model training only uses explicitly consented data

    Benefits:

    • Compliance with emerging AI governance regulations

    • Transparent AI development practices

    • Customer trust through explicit AI consent

    • Auditable training data provenance

    Comparison with Existing Solutions

    Traditional Database-Centric Approaches

    Limitations:

    • Consent stored separately from data

    • Complex joins required for compliance checking

    • Difficult to track consent across system boundaries

    • No standardization across vendors

    vCon Consent Advantages:

    • Consent embedded directly in data containers

    • Self-contained compliance verification

    • Standard format enables vendor interoperability

    • Cryptographic integrity protection

    Consent Management Platforms (CMPs)

    Limitations:

    • Focused on web-based consent (cookies, tracking)

    • Limited support for conversational data

    • No integration with AI processing workflows

    • Proprietary formats and APIs

    vCon Consent Advantages:

    • Purpose-built for conversational data

    • Native AI governance support

    • Open standard with multiple implementations

    • Cryptographic transparency and verification

    Privacy Engineering Frameworks

    Limitations:

    • Require custom implementation for each use case

    • No standardized consent representation

    • Limited audit and transparency features

    • Complex integration with existing systems

    vCon Consent Advantages:

    • Standardized consent attachment format

    • Built-in transparency and audit capabilities

    • Designed for easy integration

    • Comprehensive privacy framework support

    Implementation Guide

    Getting Started

    1. Assess Current Architecture

      • Identify conversation data storage systems

      • Map existing consent collection processes

      • Determine regulatory compliance requirements

    2. Choose Integration Approach

      • Greenfield: Build vCon-native systems from scratch

      • Migration: Gradually convert existing data to vCon format

      • Hybrid: Use vCon consent for new data while maintaining legacy systems

    3. Set Up Consent Ledger

      • Deploy SCITT-compatible transparency service

      • Configure cryptographic signing keys

      • Establish backup and recovery procedures

    4. Implement Processing Logic

      • Update data processing pipelines to check consent

      • Add consent verification to AI training workflows

      • Implement consent expiration handling

    Best Practices

    Design Principles:

    • Start with minimal viable consent categories

    • Design for consent evolution and expansion

    • Implement comprehensive audit logging

    • Plan for consent migration and versioning

    Operational Considerations:

    • Establish clear consent renewal processes

    • Monitor consent expiration and renewal rates

    • Implement automated compliance reporting

    • Train support staff on consent management procedures

    Future Directions

    Emerging Trends

    AI Governance Integration:

    • Enhanced support for AI model cards and documentation

    • Integration with algorithmic auditing frameworks

    • Support for AI explainability requirements

    Regulatory Evolution:

    • Adaptation to new privacy regulations

    • Support for sector-specific compliance requirements

    • Integration with international data transfer mechanisms

    Technical Advancement:

    • Zero-knowledge proof integration for privacy-preserving verification

    • Blockchain integration for decentralized consent management

    • Machine learning for consent anomaly detection

    Research Opportunities

    Academic Research Areas:

    • Privacy-preserving consent verification mechanisms

    • Economic models for consent-based data markets

    • User experience optimization for consent interfaces

    • Cross-border consent harmonization frameworks

    Industry Development:

    • Open-source consent ledger implementations

    • Industry-specific consent vocabulary standards

    • Integration with existing enterprise systems

    • Performance optimization for high-volume deployments

    Conclusion

    The vCon consent attachment specification represents a significant advancement in privacy-first data management. By embedding structured consent information directly within conversation containers, organizations can achieve comprehensive privacy compliance while enabling responsible AI development.

    Key Takeaways:

    1. Privacy by Design: Consent attachments ensure privacy preferences travel with data throughout its lifecycle

    2. Regulatory Compliance: Built-in support for GDPR, CCPA, HIPAA, and emerging AI governance frameworks

    3. Granular Control: Purpose-specific permissions enable nuanced consent management

    4. Cryptographic Verification: Tamper-evident records provide transparent audit trails

    5. Standard Format: Open specification enables vendor interoperability and innovation

    As privacy regulations continue to evolve and AI applications become more sophisticated, the need for robust consent management will only grow. The vCon consent attachment specification provides a foundation for building privacy-respecting systems that can adapt to future requirements while maintaining user trust and regulatory compliance.

    Organizations implementing this approach will be well-positioned to navigate the complex intersection of data utility, privacy protection, and regulatory compliance in the age of AI.


    References and Further Reading

    • IETF vCon Working Group: https://datatracker.ietf.org/wg/vcon/

    • SCITT Working Group: https://datatracker.ietf.org/wg/scitt/

    • AI Preferences Vocabulary: https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/

    • vCon Consent Specification: https://datatracker.ietf.org/doc/draft-vcon-consent/

    • GDPR Compliance Guide:

    • CCPA Resource Center:

    For questions about implementation or to contribute to the specification, contact the vCon working group at [email protected]

    {
      "type": "consent",
      "expiration": "2026-01-02T12:00:00Z",
      "party": 0,
      "dialog": 0,
      "consents": [
        {
          "purpose": "recording",
          "status": "granted",
          "timestamp": "2025-01-02T12:15:30Z"
        },
        {
          "purpose": "ai_training",
          "status": "denied",
          "vocabulary": "ai-pref",
          "timestamp": "2025-01-02T12:15:30Z"
        }
      ],
      "proof": [
        {
          "type": "verbal_confirmation",
          "timestamp": "2025-01-02T12:15:30Z"
        }
      ]
    }
    def process_conversation(vcon):
        # Extract consent attachment
        consent_attachment = vcon.get_attachment_by_type("consent")
        
        # Check if processing is allowed
        if consent_attachment.allows_purpose("transcription"):
            transcript = transcribe_audio(vcon.dialog[0])
            vcon.add_analysis(transcript)
        
        # Respect AI training preferences
        if not consent_attachment.allows_purpose("ai_training"):
            vcon.add_processing_restriction("no_ai_training")
        
        return vcon
    def verify_consent_status(consent_attachment):
        # Check local consent validity
        if consent_attachment.is_expired():
            return False
        
        # Verify with consent ledger if configured
        if consent_attachment.has_ledger():
            ledger_status = query_consent_ledger(
                consent_attachment.consent_ledger,
                consent_attachment.consent_id
            )
            return ledger_status.is_valid()
        
        return True
    https://gdpr.eu/
    https://oag.ca.gov/privacy/ccpa

    Quickstart

    The Python vCon library

    vCon Library

    A Python library for working with vCon (Virtual Conversation) objects according to the vCon specification.

    Overview

    The vCon library provides a complete implementation of the vCon format for representing conversations and related metadata. It supports all features defined in the latest vCon specification including:

    • Conversation Management: Parties, dialogs, attachments, and analysis

    • Contact Information: Multiple contact methods (tel, email, SIP, DID)

    • Media Support: Audio, video, text, and image formats

    • Security: Digital signatures and content hashing

    Quick Start

    Basic vCon Creation

    Privacy Compliance with Lawful Basis Extension

    Transcription with WTF Extension

    Complete Example with Extensions

    Key Features

    This library implements the latest vCon specification with the following features:

    Enhanced Party Information

    Extensions and Must-Support

    Enhanced Dialog Support

    Party History Events

    Disposition Values for Incomplete Dialogs

    Civic Address Support

    Lawful Basis Extension (Privacy Compliance)

    WTF Extension (Transcription Support)

    Provider Data Conversion

    Installation

    Basic Usage

    Creating a vCon

    Loading a vCon

    Validation

    Media Support

    Audio and Video

    Supported Media Types

    Audio: audio/x-wav, audio/x-mp3, audio/x-mp4, audio/ogg Video: video/x-mp4, video/ogg Text: text/plain Multipart: multipart/mixed

    Security Features

    Digital Signatures

    Content Hashing

    Advanced Features

    Property Handling

    Transfer Dialogs

    Analysis Data

    Specification Compliance

    This library implements the latest vCon specification with:

    • ✅ All required fields and validation

    • ✅ Proper media type support

    • ✅ Civic address (GEOPRIV) compliance

    • ✅ Party history event tracking

    Testing

    Run the test suite:

    All tests pass, covering:

    • Basic functionality

    • Enhanced vCon features

    • Validation and error handling

    • Media type support

    Extension Framework

    The vCon library includes a comprehensive extension framework that allows for standardized implementation of additional functionality:

    Available Extensions

    • Lawful Basis Extension - GDPR-compliant privacy management and consent tracking

    • WTF Extension - World Transcription Format for standardized speech-to-text data

    Extension Features

    • Validation Framework - Comprehensive validation for all extension data

    • Processing Framework - Standardized processing and analysis

    • Provider Adapters - Automatic conversion from provider-specific formats

    • Export Capabilities - Multiple export formats (SRT, WebVTT)

    Extension Usage

    Documentation

    • API Reference - Complete API documentation

    • User Guide - Comprehensive usage guide

    • LLM Guide - Guide for AI-assisted development

    • Migration Guide - Upgrading from older versions

    License

    This project is licensed under the MIT License - see the LICENSE file for details.

    Extensibility: Extensions and must_support fields

  • Location Data: Civic address information (GEOPRIV)

  • Event Tracking: Party history with join/drop/hold/mute events

  • Privacy Compliance: Lawful Basis extension for GDPR compliance

  • Transcription Support: WTF (World Transcription Format) extension for standardized speech-to-text

  • ✅ Transfer dialog support
  • ✅ Content hashing and security

  • ✅ Extensions and must_support

  • ✅ Flexible versioning (version field is optional)

  • ✅ Backward compatibility

  • ✅ Lawful Basis Extension - GDPR-compliant privacy management

  • ✅ WTF Extension - World Transcription Format for standardized speech-to-text

  • ✅ Extension Framework - Comprehensive validation and processing

  • ✅ Multi-Provider Support - Whisper, Deepgram, AssemblyAI, and more

  • ✅ Export Capabilities - SRT and WebVTT subtitle formats

  • Security features
  • Flexible versioning

  • Backward compatibility

  • Extension Framework - Lawful Basis and WTF extensions

  • Privacy Compliance - GDPR-compliant consent management

  • Transcription Support - Multi-provider transcription handling

  • Provider Adapters - Data conversion and validation

  • Export Functionality - SRT and WebVTT export testing

  • Permission Management - Granular permission checking and validation

    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime, timezone
    
    # Create a new vCon
    vcon = Vcon.build_new()
    
    # Add parties
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add dialog
    dialog = Dialog(
        type="text",
        start=datetime.now(timezone.utc),
        parties=[0, 1],
        body="Hello, I need help with my account."
    )
    vcon.add_dialog(dialog)
    
    # Save to file
    vcon.save_to_file("conversation.vcon.json")
    print(f"Created vCon: {vcon.uuid}")
    from datetime import timedelta
    
    # Add lawful basis for consent
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "transcription",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            }
        ],
        party_index=0
    )
    
    # Check permissions
    can_record = vcon.check_lawful_basis_permission("recording", party_index=0)
    print(f"Can record: {can_record}")
    # Add transcription
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, I need help with my account.",
            "language": "en",
            "duration": 4.2,
            "confidence": 0.92
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 4.2,
                "text": "Hello, I need help with my account.",
                "confidence": 0.92,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1"
        },
        party_index=0,
        dialog_index=0
    )
    
    # Export to SRT format
    attachments = vcon.find_wtf_attachments(party_index=0)
    if attachments:
        from vcon.extensions.wtf import WTFAttachment
        wtf_attachment = WTFAttachment.from_dict(attachments[0]["body"])
        srt_content = wtf_attachment.export_to_srt()
        print("SRT Export:")
        print(srt_content)
    # Create comprehensive vCon with extensions
    vcon = Vcon.build_new()
    
    # Add parties
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add recording dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(timezone.utc),
        parties=[0, 1],
        mimetype="audio/mp3"
    )
    vcon.add_dialog(dialog)
    
    # Add lawful basis for consent
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "transcription",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            }
        ],
        party_index=0
    )
    
    # Add transcription
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, I need help with my account.",
            "language": "en",
            "duration": 4.2,
            "confidence": 0.92
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 4.2,
                "text": "Hello, I need help with my account.",
                "confidence": 0.92,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1"
        },
        party_index=0,
        dialog_index=0
    )
    
    # Add extensions
    vcon.add_extension("lawful_basis")
    vcon.add_extension("wtf_transcription")
    
    # Validate extensions
    validation_results = vcon.validate_extensions()
    print("Extension validation:", validation_results)
    
    # Save vCon
    vcon.save_to_file("complete_conversation.vcon.json")
    print(f"Created complete vCon with extensions: {vcon.uuid}")
    from vcon import Vcon, Party
    
    # Create a party with enhanced contact information
    party = Party(
        tel="+1234567890",
        name="John Doe",
        sip="sip:[email protected]",
        did="did:example:123456789abcdef",
        jCard={
            "fn": "John Doe",
            "tel": "+1234567890",
            "email": "[email protected]"
        },
        timezone="America/New_York"
    )
    vcon = Vcon.build_new()
    
    # Add extensions used in this vCon
    vcon.add_extension("video")
    vcon.add_extension("encryption")
    
    # Add extensions that must be supported
    vcon.add_must_support("encryption")
    
    print(vcon.get_extensions())  # ['video', 'encryption']
    print(vcon.get_must_support())  # ['encryption']
    from vcon import Dialog
    from datetime import datetime
    
    # Create dialog with new fields
    dialog = Dialog(
        type="text",
        start=datetime.now(),
        parties=[0, 1],
        session_id="session-12345",
        content_hash="c8d3d67f662a787e96e74ccb0a77803138c0f13495a186ccbde495c57c385608",
        application="chat-app",
        message_id="<[email protected]>"
    )
    from vcon import PartyHistory
    from datetime import datetime
    
    # Track party events
    history = [
        PartyHistory(0, "join", datetime.now()),
        PartyHistory(1, "join", datetime.now()),
        PartyHistory(0, "hold", datetime.now()),
        PartyHistory(0, "unhold", datetime.now()),
        PartyHistory(1, "drop", datetime.now())
    ]
    # Create incomplete dialog with proper disposition
    incomplete_dialog = Dialog(
        type="incomplete",
        start=datetime.now(),
        parties=[0],
        disposition="no-answer"  # Valid: no-answer, congestion, failed, busy, hung-up, voicemail-no-message
    )
    from vcon import CivicAddress
    
    # Create civic address with GEOPRIV fields
    address = CivicAddress(
        country="US",
        a1="CA",
        a3="San Francisco",
        sts="Market Street",
        hno="123",
        pc="94102"
    )
    
    party = Party(name="Jane", civicaddress=address)
    from datetime import datetime, timezone, timedelta
    
    # Add lawful basis for GDPR compliance
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "analysis",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat(),
                "conditions": ["anonymized_data_only"]
            }
        ],
        party_index=0
    )
    
    # Check permissions
    can_record = vcon.check_lawful_basis_permission("recording", party_index=0)
    can_analyze = vcon.check_lawful_basis_permission("analysis", party_index=0)
    
    # Find lawful basis attachments
    attachments = vcon.find_lawful_basis_attachments(party_index=0)
    # Add standardized transcription
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, this is a test transcription.",
            "language": "en",
            "duration": 3.5,
            "confidence": 0.95
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 1.5,
                "text": "Hello, this is",
                "confidence": 0.95,
                "speaker": 0
            },
            {
                "id": 1,
                "start": 1.5,
                "end": 3.5,
                "text": "a test transcription.",
                "confidence": 0.94,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1"
        },
        party_index=0,
        dialog_index=0
    )
    
    # Export to subtitle formats
    attachments = vcon.find_wtf_attachments(party_index=0)
    if attachments:
        from vcon.extensions.wtf import WTFAttachment
        wtf_attachment = WTFAttachment.from_dict(attachments[0]["body"])
        
        # Export to SRT format
        srt_content = wtf_attachment.export_to_srt()
        
        # Export to WebVTT format
        vtt_content = wtf_attachment.export_to_vtt()
    from vcon.extensions.wtf import WhisperAdapter, DeepgramAdapter
    
    # Convert Whisper data to WTF format
    whisper_data = {
        "text": "Hello world from Whisper",
        "segments": [
            {
                "start": 0.0,
                "end": 2.0,
                "text": "Hello world from Whisper"
            }
        ]
    }
    
    whisper_adapter = WhisperAdapter()
    wtf_attachment = whisper_adapter.convert(whisper_data)
    
    # Add to vCon
    vcon.add_wtf_transcription_attachment(
        transcript=wtf_attachment.transcript.to_dict(),
        segments=[segment.to_dict() for segment in wtf_attachment.segments],
        metadata=wtf_attachment.metadata.to_dict()
    )
    pip install vcon
    from vcon import Vcon, Party, Dialog
    from datetime import datetime
    
    # Create a new vCon
    vcon = Vcon.build_new()
    
    # Add parties
    alice = Party(tel="+1234567890", name="Alice", role="caller")
    bob = Party(tel="+1987654321", name="Bob", role="agent")
    
    vcon.add_party(alice)
    vcon.add_party(bob)
    
    # Add dialog
    dialog = Dialog(
        type="text",
        start=datetime.now(),
        parties=[0, 1],
        body="Hello, this is a test message!"
    )
    
    vcon.add_dialog(dialog)
    
    # Save to file
    vcon.save_to_file("conversation.vcon.json")
    # Load from file
    vcon = Vcon.load("conversation.vcon.json")
    
    # Load from URL
    vcon = Vcon.load("https://example.com/conversation.vcon.json")
    # Validate a vCon
    is_valid, errors = vcon.is_valid()
    
    if is_valid:
        print("vCon is valid")
    else:
        print("Validation errors:", errors)
    
    # Validate from file
    is_valid, errors = Vcon.validate_file("conversation.vcon.json")
    # Add audio recording
    audio_dialog = Dialog(
        type="recording",
        start=datetime.now(),
        parties=[0, 1],
        url="https://example.com/recording.wav",
        mimetype="audio/x-wav"
    )
    
    # Add video with metadata
    video_dialog = Dialog(
        type="video",
        start=datetime.now(),
        parties=[0, 1],
        url="https://example.com/video.mp4",
        mimetype="video/mp4",
        resolution="1920x1080",
        frame_rate=30.0,
        codec="H.264"
    )
    from cryptography.hazmat.primitives import serialization
    
    # Generate key pair
    private_key, public_key = Vcon.generate_key_pair()
    
    # Sign the vCon
    vcon.sign(private_key)
    
    # Verify signature
    is_valid = vcon.verify(public_key)
    # Calculate content hash for external files
    content_hash = dialog.calculate_content_hash("sha256")
    
    # Verify content integrity
    is_valid = dialog.verify_content_hash(expected_hash, "sha256")
    # Strict mode - only allow standard properties
    vcon = Vcon.load("file.json", property_handling="strict")
    
    # Meta mode - move non-standard properties to meta object
    vcon = Vcon.load("file.json", property_handling="meta")
    
    # Default mode - keep all properties
    vcon = Vcon.load("file.json", property_handling="default")
    # Create transfer dialog
    transfer_data = {
        "transferee": 0,
        "transferor": 1,
        "transfer_target": 2,
        "original": 0,
        "target_dialog": 1
    }
    
    vcon.add_transfer_dialog(
        start=datetime.now(),
        transfer_data=transfer_data,
        parties=[0, 1, 2]
    )
    # Add analysis
    vcon.add_analysis(
        type="sentiment",
        dialog=0,
        vendor="example-vendor",
        body={"sentiment": "positive", "confidence": 0.95},
        encoding="json"
    )
    pytest tests/
    # Validate all extensions
    validation_results = vcon.validate_extensions()
    
    # Process all extensions
    processing_results = vcon.process_extensions()
    
    # Check specific permissions
    can_record = vcon.check_lawful_basis_permission("recording", party_index=0)
    
    # Export transcriptions
    attachments = vcon.find_wtf_attachments(party_index=0)

    vCon Adapter Development Guide

    A comprehensive guide for building adapters that convert conversation data from various systems into the standardized vCon format using the vCon Python library.

    Table of Contents

    1. Introduction to vCon Adapters

    2. Adapter Architecture

    Introduction to vCon Adapters

    vCon adapters are specialized components that transform conversation data from various sources (call centers, chat systems, video conferencing platforms, etc.) into the standardized vCon format. This enables interoperability between different conversation systems and provides a unified way to store, analyze, and exchange conversation data.

    What vCon Adapters Do

    • Extract conversation data from source systems

    • Transform data into vCon-compliant structures

    • Validate the resulting vCon objects

    • Export vCons for storage or further processing

    Common Use Cases

    • Call Center Integration: Convert PBX call records to vCons

    • Chat Platform Export: Transform Slack/Teams conversations

    • Video Conference Processing: Convert Zoom/WebEx recordings

    • Email Threading: Convert email chains to conversation format

    Adapter Architecture

    Basic Adapter Structure

    Modular Design Pattern

    Getting Started

    Installation and Setup

    Simple Adapter Example

    Core Adapter Components

    1. Data Extractors

    2. Party Mappers

    3. Dialog Processors

    Data Mapping Strategies

    1. Configuration-Driven Mapping

    2. Schema-Based Transformation

    Media Handling

    1. Audio Processing

    2. Video Processing

    Best Practices

    1. Error Handling and Resilience

    2. Performance Optimization

    3. Logging and Monitoring

    Testing Your Adapter

    1. Unit Testing Framework

    2. Property-Based Testing

    Common Patterns

    1. Multi-Source Adapter

    2. Streaming Adapter

    3. Incremental Adapter

    Troubleshooting

    Common Issues and Solutions

    1. Invalid vCon Generation

    Problem: Generated vCons fail validation

    Solutions:

    2. Memory Issues with Large Conversations

    Problem: Large conversations cause memory issues

    Solutions:

    3. Character Encoding Issues

    Problem: Non-UTF-8 characters in source data

    Solutions:

    Testing and Validation Tools


    This guide provides a comprehensive foundation for building robust vCon adapters. The patterns and examples can be adapted for specific use cases and source systems while maintaining compliance with the vCon specification.

    Social Media Monitoring: Transform social interactions

    Getting Started
    Core Adapter Components
    Data Mapping Strategies
    Media Handling
    Best Practices
    Testing Your Adapter
    Common Patterns
    Troubleshooting
    from abc import ABC, abstractmethod
    from typing import Dict, List, Any, Optional
    from vcon import Vcon, Party, Dialog
    from datetime import datetime
    
    class BaseVconAdapter(ABC):
        """Base class for all vCon adapters."""
        
        def __init__(self, config: Dict[str, Any]):
            self.config = config
            self.validation_errors = []
        
        @abstractmethod
        def extract_data(self, source: Any) -> Dict[str, Any]:
            """Extract raw data from the source system."""
            pass
        
        @abstractmethod
        def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
            """Transform raw data into a vCon object."""
            pass
        
        def validate_vcon(self, vcon: Vcon) -> bool:
            """Validate the generated vCon."""
            is_valid, errors = vcon.is_valid()
            self.validation_errors = errors
            return is_valid
        
        def process(self, source: Any) -> Vcon:
            """Main processing pipeline."""
            raw_data = self.extract_data(source)
            vcon = self.transform_to_vcon(raw_data)
            
            if not self.validate_vcon(vcon):
                raise ValueError(f"Invalid vCon generated: {self.validation_errors}")
            
            return vcon
    class VconAdapterPipeline:
        """Pipeline for processing conversation data through multiple stages."""
        
        def __init__(self):
            self.extractors = []
            self.transformers = []
            self.validators = []
            self.exporters = []
        
        def add_extractor(self, extractor):
            self.extractors.append(extractor)
        
        def add_transformer(self, transformer):
            self.transformers.append(transformer)
        
        def process_conversation(self, source_data):
            # Extract
            for extractor in self.extractors:
                source_data = extractor.process(source_data)
            
            # Transform
            vcon = Vcon.build_new()
            for transformer in self.transformers:
                vcon = transformer.apply(vcon, source_data)
            
            # Validate
            for validator in self.validators:
                validator.validate(vcon)
            
            return vcon
    # Install the vCon library
    pip install vcon
    
    # Basic adapter setup
    from vcon import Vcon, Party, Dialog
    import json
    from datetime import datetime
    class ChatLogAdapter(BaseVconAdapter):
        """Adapter for converting chat logs to vCon format."""
        
        def extract_data(self, chat_file_path: str) -> Dict[str, Any]:
            """Extract data from a chat log file."""
            with open(chat_file_path, 'r') as f:
                return json.load(f)
        
        def transform_to_vcon(self, raw_data: Dict[str, Any]) -> Vcon:
            """Transform chat data to vCon."""
            vcon = Vcon.build_new()
            
            # Add metadata
            vcon.add_tag("source", "chat_log")
            vcon.add_tag("platform", raw_data.get("platform", "unknown"))
            
            # Process participants
            participant_map = {}
            for i, participant in enumerate(raw_data.get("participants", [])):
                party = Party(
                    name=participant["name"],
                    mailto=participant.get("email"),
                    role=participant.get("role", "participant")
                )
                vcon.add_party(party)
                participant_map[participant["id"]] = i
            
            # Process messages
            for message in raw_data.get("messages", []):
                dialog = Dialog(
                    type="text",
                    start=datetime.fromisoformat(message["timestamp"]),
                    parties=[participant_map[message["sender_id"]]],
                    originator=participant_map[message["sender_id"]],
                    mimetype="text/plain",
                    body=message["content"]
                )
                vcon.add_dialog(dialog)
            
            return vcon
    
    # Usage
    adapter = ChatLogAdapter({})
    vcon = adapter.process("chat_log.json")
    vcon.save_to_file("conversation.vcon.json")
    class CallRecordExtractor:
        """Extract data from call center records."""
        
        def __init__(self, api_client):
            self.api_client = api_client
        
        def extract_call_data(self, call_id: str) -> Dict[str, Any]:
            """Extract call data from the system."""
            call_record = self.api_client.get_call(call_id)
            recording_url = self.api_client.get_recording_url(call_id)
            transcript = self.api_client.get_transcript(call_id)
            
            return {
                "call_record": call_record,
                "recording_url": recording_url,
                "transcript": transcript,
                "metadata": self.api_client.get_call_metadata(call_id)
            }
    
    class EmailThreadExtractor:
        """Extract data from email threads."""
        
        def extract_thread(self, thread_id: str) -> Dict[str, Any]:
            """Extract email thread data."""
            # Implementation for email extraction
            pass
    class PartyMapper:
        """Maps source system participants to vCon parties."""
        
        def __init__(self, mapping_rules: Dict[str, str]):
            self.mapping_rules = mapping_rules
        
        def map_party(self, source_participant: Dict[str, Any]) -> Party:
            """Map a source participant to a vCon Party."""
            return Party(
                name=source_participant.get(self.mapping_rules.get("name", "name")),
                tel=source_participant.get(self.mapping_rules.get("phone", "phone")),
                mailto=source_participant.get(self.mapping_rules.get("email", "email")),
                role=self.determine_role(source_participant),
                uuid=source_participant.get("id")
            )
        
        def determine_role(self, participant: Dict[str, Any]) -> str:
            """Determine the role of a participant."""
            if participant.get("is_agent"):
                return "agent"
            elif participant.get("is_customer"):
                return "customer"
            else:
                return "participant"
    class DialogProcessor:
        """Process different types of dialog content."""
        
        def process_text_message(self, message: Dict[str, Any], parties_map: Dict) -> Dialog:
            """Process a text message into a Dialog."""
            return Dialog(
                type="text",
                start=self.parse_timestamp(message["timestamp"]),
                parties=self.get_involved_parties(message, parties_map),
                originator=parties_map[message["sender_id"]],
                mimetype="text/plain",
                body=message["content"]
            )
        
        def process_audio_recording(self, recording: Dict[str, Any], parties_map: Dict) -> Dialog:
            """Process an audio recording into a Dialog."""
            return Dialog(
                type="recording",
                start=self.parse_timestamp(recording["start_time"]),
                parties=list(parties_map.values()),
                mimetype="audio/wav",
                url=recording["url"],
                duration=recording.get("duration")
            )
        
        def process_video_call(self, video_data: Dict[str, Any], parties_map: Dict) -> Dialog:
            """Process a video call into a Dialog."""
            return Dialog(
                type="video",
                start=self.parse_timestamp(video_data["start_time"]),
                parties=list(parties_map.values()),
                mimetype="video/mp4",
                url=video_data["recording_url"],
                resolution=video_data.get("resolution", "1920x1080"),
                duration=video_data.get("duration")
            )
    class ConfigurableMappingAdapter(BaseVconAdapter):
        """Adapter with configurable field mappings."""
        
        def __init__(self, config: Dict[str, Any]):
            super().__init__(config)
            self.field_mappings = config.get("field_mappings", {})
            self.party_mappings = config.get("party_mappings", {})
            self.dialog_mappings = config.get("dialog_mappings", {})
        
        def map_field(self, source_data: Dict, mapping_path: str, default=None):
            """Map a field from source data using configured path."""
            keys = mapping_path.split(".")
            value = source_data
            
            try:
                for key in keys:
                    value = value[key]
                return value
            except (KeyError, TypeError):
                return default
    
    # Configuration example
    mapping_config = {
        "field_mappings": {
            "conversation_id": "call.id",
            "start_time": "call.started_at",
            "end_time": "call.ended_at"
        },
        "party_mappings": {
            "name": "participant.display_name",
            "phone": "participant.phone_number",
            "email": "participant.email_address"
        },
        "dialog_mappings": {
            "timestamp": "message.created_at",
            "content": "message.body",
            "sender": "message.from"
        }
    }
    from jsonschema import validate
    
    class SchemaBasedAdapter(BaseVconAdapter):
        """Adapter that validates input against a schema before transformation."""
        
        def __init__(self, config: Dict[str, Any]):
            super().__init__(config)
            self.input_schema = config["input_schema"]
            self.transformation_rules = config["transformation_rules"]
        
        def extract_data(self, source: Dict[str, Any]) -> Dict[str, Any]:
            """Extract and validate data against schema."""
            validate(source, self.input_schema)
            return source
        
        def apply_transformation_rules(self, data: Dict[str, Any]) -> Dict[str, Any]:
            """Apply transformation rules to normalize data."""
            transformed = {}
            
            for target_field, rule in self.transformation_rules.items():
                if rule["type"] == "direct_map":
                    transformed[target_field] = data.get(rule["source_field"])
                elif rule["type"] == "function":
                    transformed[target_field] = self.apply_function(
                        rule["function"], 
                        data.get(rule["source_field"])
                    )
            
            return transformed
    class AudioMediaHandler:
        """Handle audio content in conversations."""
        
        def process_audio_dialog(self, audio_data: Dict[str, Any]) -> Dialog:
            """Process audio data into a vCon dialog."""
            if audio_data.get("is_external"):
                return self.create_external_audio_dialog(audio_data)
            else:
                return self.create_inline_audio_dialog(audio_data)
        
        def create_external_audio_dialog(self, audio_data: Dict[str, Any]) -> Dialog:
            """Create dialog with external audio reference."""
            return Dialog(
                type="recording",
                start=audio_data["start_time"],
                parties=audio_data["participants"],
                mimetype=self.detect_audio_mimetype(audio_data["url"]),
                url=audio_data["url"],
                duration=audio_data.get("duration"),
                content_hash=audio_data.get("checksum")
            )
        
        def create_inline_audio_dialog(self, audio_data: Dict[str, Any]) -> Dialog:
            """Create dialog with inline audio data."""
            import base64
            
            # Read audio file and encode
            with open(audio_data["file_path"], "rb") as f:
                audio_bytes = f.read()
            
            encoded_audio = base64.b64encode(audio_bytes).decode()
            
            return Dialog(
                type="recording",
                start=audio_data["start_time"],
                parties=audio_data["participants"],
                mimetype=self.detect_audio_mimetype(audio_data["file_path"]),
                body=encoded_audio,
                encoding="base64",
                filename=audio_data.get("filename")
            )
    class VideoMediaHandler:
        """Handle video content in conversations."""
        
        def process_video_call(self, video_data: Dict[str, Any]) -> Dialog:
            """Process video call data."""
            dialog = Dialog(
                type="video",
                start=video_data["start_time"],
                parties=video_data["participants"],
                mimetype="video/mp4",
                url=video_data["recording_url"],
                resolution=video_data.get("resolution"),
                frame_rate=video_data.get("fps"),
                codec=video_data.get("codec")
            )
            
            # Add video metadata
            if video_data.get("has_screen_share"):
                dialog.metadata = dialog.metadata or {}
                dialog.metadata["screen_share"] = True
            
            return dialog
        
        def generate_thumbnail(self, video_dialog: Dialog) -> str:
            """Generate thumbnail for video dialog."""
            if hasattr(video_dialog, 'generate_thumbnail'):
                thumbnail_data = video_dialog.generate_thumbnail(
                    timestamp=5.0,  # 5 seconds into video
                    width=320,
                    height=240
                )
                return base64.b64encode(thumbnail_data).decode()
            return None
    class ResilientAdapter(BaseVconAdapter):
        """Adapter with comprehensive error handling."""
        
        def __init__(self, config: Dict[str, Any]):
            super().__init__(config)
            self.retry_attempts = config.get("retry_attempts", 3)
            self.error_handlers = {}
        
        def register_error_handler(self, error_type: type, handler):
            """Register custom error handlers."""
            self.error_handlers[error_type] = handler
        
        def safe_extract_data(self, source: Any) -> Dict[str, Any]:
            """Extract data with error handling and retries."""
            for attempt in range(self.retry_attempts):
                try:
                    return self.extract_data(source)
                except Exception as e:
                    if type(e) in self.error_handlers:
                        return self.error_handlers[type(e)](source, e, attempt)
                    
                    if attempt == self.retry_attempts - 1:
                        raise
                    
                    # Exponential backoff
                    time.sleep(2 ** attempt)
        
        def partial_transform_with_fallbacks(self, raw_data: Dict[str, Any]) -> Vcon:
            """Transform data with fallbacks for missing fields."""
            vcon = Vcon.build_new()
            
            try:
                # Attempt full transformation
                return self.transform_to_vcon(raw_data)
            except Exception as e:
                # Fall back to partial transformation
                return self.create_minimal_vcon(raw_data, e)
        
        def create_minimal_vcon(self, raw_data: Dict[str, Any], error: Exception) -> Vcon:
            """Create minimal vCon when full transformation fails."""
            vcon = Vcon.build_new()
            
            # Add error information
            vcon.add_tag("transformation_error", str(error))
            vcon.add_tag("partial_conversion", "true")
            
            # Add raw data as attachment if possible
            try:
                vcon.add_attachment(
                    type="raw_source_data",
                    body=json.dumps(raw_data),
                    encoding="json"
                )
            except:
                pass  # Silently fail if raw data can't be serialized
            
            return vcon
    class PerformantAdapter(BaseVconAdapter):
        """Adapter optimized for performance."""
        
        def __init__(self, config: Dict[str, Any]):
            super().__init__(config)
            self.batch_size = config.get("batch_size", 100)
            self.use_threading = config.get("use_threading", False)
            self.cache_enabled = config.get("cache_enabled", True)
            self._cache = {} if self.cache_enabled else None
        
        def process_batch(self, sources: List[Any]) -> List[Vcon]:
            """Process multiple conversations in batch."""
            if self.use_threading:
                return self._process_threaded_batch(sources)
            else:
                return [self.process(source) for source in sources]
        
        def _process_threaded_batch(self, sources: List[Any]) -> List[Vcon]:
            """Process batch using threading for I/O bound operations."""
            from concurrent.futures import ThreadPoolExecutor, as_completed
            
            vcons = []
            with ThreadPoolExecutor(max_workers=5) as executor:
                future_to_source = {
                    executor.submit(self.process, source): source 
                    for source in sources
                }
                
                for future in as_completed(future_to_source):
                    try:
                        vcon = future.result()
                        vcons.append(vcon)
                    except Exception as e:
                        source = future_to_source[future]
                        print(f"Error processing {source}: {e}")
            
            return vcons
        
        def cached_lookup(self, key: str, fetch_func):
            """Cache expensive lookups."""
            if not self.cache_enabled:
                return fetch_func()
            
            if key not in self._cache:
                self._cache[key] = fetch_func()
            
            return self._cache[key]
    import logging
    from datetime import datetime
    
    class MonitoredAdapter(BaseVconAdapter):
        """Adapter with comprehensive logging and monitoring."""
        
        def __init__(self, config: Dict[str, Any]):
            super().__init__(config)
            self.logger = logging.getLogger(f"{self.__class__.__name__}")
            self.metrics = {
                "processed_count": 0,
                "error_count": 0,
                "start_time": datetime.now()
            }
        
        def process(self, source: Any) -> Vcon:
            """Process with monitoring."""
            start_time = datetime.now()
            
            try:
                self.logger.info(f"Starting processing for source: {source}")
                vcon = super().process(source)
                
                # Update metrics
                self.metrics["processed_count"] += 1
                processing_time = (datetime.now() - start_time).total_seconds()
                
                self.logger.info(
                    f"Successfully processed source in {processing_time:.2f}s"
                )
                
                return vcon
                
            except Exception as e:
                self.metrics["error_count"] += 1
                self.logger.error(f"Failed to process source: {e}", exc_info=True)
                raise
        
        def get_metrics(self) -> Dict[str, Any]:
            """Get adapter performance metrics."""
            runtime = (datetime.now() - self.metrics["start_time"]).total_seconds()
            
            return {
                **self.metrics,
                "runtime_seconds": runtime,
                "success_rate": (
                    self.metrics["processed_count"] / 
                    (self.metrics["processed_count"] + self.metrics["error_count"])
                    if (self.metrics["processed_count"] + self.metrics["error_count"]) > 0
                    else 0
                )
            }
    import unittest
    from unittest.mock import Mock, patch
    import tempfile
    import json
    
    class TestChatLogAdapter(unittest.TestCase):
        """Test suite for ChatLogAdapter."""
        
        def setUp(self):
            """Set up test fixtures."""
            self.adapter = ChatLogAdapter({})
            
            self.sample_data = {
                "platform": "slack",
                "participants": [
                    {"id": "user1", "name": "Alice", "email": "[email protected]"},
                    {"id": "user2", "name": "Bob", "email": "[email protected]"}
                ],
                "messages": [
                    {
                        "sender_id": "user1",
                        "timestamp": "2023-01-01T10:00:00Z",
                        "content": "Hello!"
                    },
                    {
                        "sender_id": "user2",
                        "timestamp": "2023-01-01T10:01:00Z",
                        "content": "Hi there!"
                    }
                ]
            }
        
        def test_extract_data(self):
            """Test data extraction from file."""
            with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
                json.dump(self.sample_data, f)
                temp_path = f.name
            
            try:
                extracted = self.adapter.extract_data(temp_path)
                self.assertEqual(extracted, self.sample_data)
            finally:
                os.unlink(temp_path)
        
        def test_transform_to_vcon(self):
            """Test transformation to vCon format."""
            vcon = self.adapter.transform_to_vcon(self.sample_data)
            
            # Verify basic structure
            self.assertEqual(len(vcon.parties), 2)
            self.assertEqual(len(vcon.dialog), 2)
            
            # Verify parties
            self.assertEqual(vcon.parties[0].name, "Alice")
            self.assertEqual(vcon.parties[1].name, "Bob")
            
            # Verify dialogs
            self.assertEqual(vcon.dialog[0]["type"], "text")
            self.assertEqual(vcon.dialog[0]["body"], "Hello!")
        
        def test_invalid_data_handling(self):
            """Test handling of invalid input data."""
            invalid_data = {"invalid": "structure"}
            
            with self.assertRaises(KeyError):
                self.adapter.transform_to_vcon(invalid_data)
        
        def test_vcon_validation(self):
            """Test that generated vCons are valid."""
            vcon = self.adapter.transform_to_vcon(self.sample_data)
            is_valid, errors = vcon.is_valid()
            
            self.assertTrue(is_valid, f"vCon validation failed: {errors}")
    
    class TestAdapterIntegration(unittest.TestCase):
        """Integration tests for adapter functionality."""
        
        def test_end_to_end_processing(self):
            """Test complete processing pipeline."""
            # Create test data
            with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
                json.dump(sample_chat_data, f)
                input_path = f.name
            
            try:
                # Process through adapter
                adapter = ChatLogAdapter({})
                vcon = adapter.process(input_path)
                
                # Save and reload to test serialization
                with tempfile.NamedTemporaryFile(mode='w', suffix='.vcon.json', delete=False) as f:
                    output_path = f.name
                
                vcon.save_to_file(output_path)
                reloaded_vcon = Vcon.load_from_file(output_path)
                
                # Verify roundtrip
                self.assertEqual(vcon.to_json(), reloaded_vcon.to_json())
                
            finally:
                os.unlink(input_path)
                os.unlink(output_path)
    from hypothesis import given, strategies as st
    
    class TestAdapterProperties(unittest.TestCase):
        """Property-based tests for adapter behavior."""
        
        @given(st.dictionaries(
            st.text(min_size=1),
            st.one_of(st.text(), st.integers(), st.floats()),
            min_size=1
        ))
        def test_adapter_handles_arbitrary_metadata(self, metadata):
            """Test that adapter handles arbitrary metadata gracefully."""
            adapter = MetadataAdapter({})
            
            try:
                # Should not crash on arbitrary input
                result = adapter.process_metadata(metadata)
                # Result should be valid JSON serializable
                json.dumps(result)
            except Exception as e:
                # If it fails, it should fail gracefully
                self.assertIsInstance(e, (ValueError, TypeError))
        
        @given(st.lists(
            st.dictionaries(
                st.sampled_from(["name", "email", "phone"]),
                st.text(min_size=1),
                min_size=1
            ),
            min_size=1,
            max_size=10
        ))
        def test_party_mapping_preserves_count(self, participants):
            """Test that party mapping preserves participant count."""
            adapter = PartyMappingAdapter({})
            mapped_parties = adapter.map_participants(participants)
            
            self.assertEqual(len(participants), len(mapped_parties))
    class MultiSourceAdapter:
        """Adapter that combines data from multiple sources."""
        
        def __init__(self, source_adapters: Dict[str, BaseVconAdapter]):
            self.source_adapters = source_adapters
        
        def process_combined(self, sources: Dict[str, Any]) -> Vcon:
            """Process data from multiple sources into a single vCon."""
            base_vcon = Vcon.build_new()
            
            # Process each source
            for source_name, source_data in sources.items():
                adapter = self.source_adapters[source_name]
                source_vcon = adapter.process(source_data)
                
                # Merge into base vCon
                base_vcon = self.merge_vcons(base_vcon, source_vcon, source_name)
            
            return base_vcon
        
        def merge_vcons(self, base: Vcon, source: Vcon, source_name: str) -> Vcon:
            """Merge source vCon into base vCon."""
            # Merge parties (avoiding duplicates)
            party_offset = len(base.parties)
            for party in source.parties:
                base.add_party(party)
            
            # Merge dialogs (adjusting party references)
            for dialog_dict in source.dialog:
                # Adjust party references
                if "parties" in dialog_dict:
                    dialog_dict["parties"] = [
                        p + party_offset for p in dialog_dict["parties"]
                    ]
                if "originator" in dialog_dict:
                    dialog_dict["originator"] += party_offset
                
                # Add source tag
                dialog = Dialog(**dialog_dict)
                dialog.metadata = dialog.metadata or {}
                dialog.metadata["source"] = source_name
                
                base.add_dialog(dialog)
            
            return base
    class StreamingAdapter:
        """Adapter for processing streaming conversation data."""
        
        def __init__(self, config: Dict[str, Any]):
            self.config = config
            self.conversation_buffer = {}
            self.buffer_timeout = config.get("buffer_timeout", 300)  # 5 minutes
        
        def process_stream_event(self, event: Dict[str, Any]) -> Optional[Vcon]:
            """Process a single streaming event."""
            conversation_id = event.get("conversation_id")
            
            if conversation_id not in self.conversation_buffer:
                self.conversation_buffer[conversation_id] = {
                    "events": [],
                    "last_updated": datetime.now()
                }
            
            # Add event to buffer
            self.conversation_buffer[conversation_id]["events"].append(event)
            self.conversation_buffer[conversation_id]["last_updated"] = datetime.now()
            
            # Check if conversation is complete
            if self.is_conversation_complete(conversation_id, event):
                return self.finalize_conversation(conversation_id)
            
            return None
        
        def is_conversation_complete(self, conversation_id: str, event: Dict[str, Any]) -> bool:
            """Determine if a conversation is complete."""
            return event.get("type") == "conversation_ended"
        
        def finalize_conversation(self, conversation_id: str) -> Vcon:
            """Convert buffered events to a vCon."""
            events = self.conversation_buffer[conversation_id]["events"]
            
            # Build vCon from events
            vcon = Vcon.build_new()
            vcon.add_tag("conversation_id", conversation_id)
            
            # Process events in chronological order
            for event in sorted(events, key=lambda e: e.get("timestamp", "")):
                self.process_event_to_vcon(vcon, event)
            
            # Clean up buffer
            del self.conversation_buffer[conversation_id]
            
            return vcon
        
        def cleanup_stale_conversations(self):
            """Clean up conversations that have been buffered too long."""
            now = datetime.now()
            stale_conversations = [
                conv_id for conv_id, data in self.conversation_buffer.items()
                if (now - data["last_updated"]).seconds > self.buffer_timeout
            ]
            
            for conv_id in stale_conversations:
                # Force finalize stale conversations
                self.finalize_conversation(conv_id)
    class IncrementalAdapter:
        """Adapter that processes conversations incrementally."""
        
        def __init__(self, config: Dict[str, Any]):
            self.config = config
            self.checkpoint_manager = CheckpointManager(config.get("checkpoint_file"))
        
        def process_incremental(self, source_iterator) -> Iterator[Vcon]:
            """Process conversations incrementally with checkpointing."""
            last_checkpoint = self.checkpoint_manager.get_last_checkpoint()
            
            for item in source_iterator:
                # Skip items already processed
                if self.is_before_checkpoint(item, last_checkpoint):
                    continue
                
                try:
                    vcon = self.process_item(item)
                    yield vcon
                    
                    # Update checkpoint
                    self.checkpoint_manager.update_checkpoint(
                        self.get_item_checkpoint(item)
                    )
                    
                except Exception as e:
                    # Log error and continue
                    self.handle_processing_error(item, e)
        
        def process_item(self, item: Any) -> Vcon:
            """Process a single item to vCon."""
            # Implementation specific to source format
            pass
        
        def is_before_checkpoint(self, item: Any, checkpoint: Any) -> bool:
            """Check if item was processed in previous run."""
            # Implementation specific to source format
            pass
    
    class CheckpointManager:
        """Manage processing checkpoints for incremental processing."""
        
        def __init__(self, checkpoint_file: str):
            self.checkpoint_file = checkpoint_file
        
        def get_last_checkpoint(self) -> Any:
            """Get the last processing checkpoint."""
            try:
                with open(self.checkpoint_file, 'r') as f:
                    return json.load(f)
            except FileNotFoundError:
                return None
        
        def update_checkpoint(self, checkpoint: Any):
            """Update the processing checkpoint."""
            with open(self.checkpoint_file, 'w') as f:
                json.dump(checkpoint, f)
    def debug_vcon_validation(vcon: Vcon):
        """Debug vCon validation issues."""
        is_valid, errors = vcon.is_valid()
        
        if not is_valid:
            print("Validation errors:")
            for error in errors:
                print(f"  - {error}")
            
            # Check specific common issues
            if not vcon.parties:
                print("  Issue: No parties defined")
            
            if not vcon.dialog:
                print("  Issue: No dialog entries")
            
            for i, dialog in enumerate(vcon.dialog):
                if "parties" in dialog:
                    max_party_idx = max(dialog["parties"]) if dialog["parties"] else -1
                    if max_party_idx >= len(vcon.parties):
                        print(f"  Issue: Dialog {i} references invalid party index {max_party_idx}")
    class StreamingVconWriter:
        """Write vCons in streaming fashion to handle large conversations."""
        
        def __init__(self, output_file: str):
            self.output_file = output_file
            self.base_vcon = None
        
        def initialize_vcon(self, metadata: Dict[str, Any]):
            """Initialize base vCon structure."""
            self.base_vcon = Vcon.build_new()
            # Add metadata, parties, etc.
        
        def add_dialog_batch(self, dialogs: List[Dialog]):
            """Add dialogs in batches to manage memory."""
            for dialog in dialogs:
                self.base_vcon.add_dialog(dialog)
            
            # Periodically flush to disk if needed
            if len(self.base_vcon.dialog) > 1000:
                self.flush_to_disk()
        
        def flush_to_disk(self):
            """Flush current vCon to disk."""
            self.base_vcon.save_to_file(self.output_file)
    def safe_decode_text(raw_text: bytes, fallback_encoding: str = "latin-1") -> str:
        """Safely decode text with fallback encoding."""
        try:
            return raw_text.decode("utf-8")
        except UnicodeDecodeError:
            try:
                return raw_text.decode(fallback_encoding)
            except UnicodeDecodeError:
                # Last resort: replace invalid characters
                return raw_text.decode("utf-8", errors="replace")
    
    def sanitize_content(content: str) -> str:
        """Sanitize content for vCon compatibility."""
        # Remove null bytes and other problematic characters
        content = content.replace("\x00", "")
        
        # Normalize unicode
        import unicodedata
        content = unicodedata.normalize("NFKC", content)
        
        return content
    class AdapterTestSuite:
        """Comprehensive test suite for vCon adapters."""
        
        def __init__(self, adapter: BaseVconAdapter):
            self.adapter = adapter
        
        def run_full_test_suite(self, test_data: List[Any]) -> Dict[str, Any]:
            """Run complete test suite on adapter."""
            results = {
                "total_tests": len(test_data),
                "successful": 0,
                "failed": 0,
                "errors": []
            }
            
            for i, data in enumerate(test_data):
                try:
                    vcon = self.adapter.process(data)
                    
                    # Validate vCon
                    is_valid, errors = vcon.is_valid()
                    if is_valid:
                        results["successful"] += 1
                    else:
                        results["failed"] += 1
                        results["errors"].append(f"Test {i}: {errors}")
                        
                except Exception as e:
                    results["failed"] += 1
                    results["errors"].append(f"Test {i}: Exception {e}")
            
            return results
        
        def benchmark_adapter(self, test_data: List[Any]) -> Dict[str, float]:
            """Benchmark adapter performance."""
            import time
            
            start_time = time.time()
            
            for data in test_data:
                self.adapter.process(data)
            
            end_time = time.time()
            
            total_time = end_time - start_time
            avg_time = total_time / len(test_data)
            
            return {
                "total_time": total_time,
                "average_time_per_item": avg_time,
                "items_per_second": len(test_data) / total_time
            }

    Library API Reference

    Complete API documentation for the vCon library - a Python implementation of the vCon 0.3.0 specification for Virtual Conversation objects.

    vCon Library API Reference

    Complete API documentation for the vCon library - a Python implementation of the latest vCon specification for Virtual Conversation objects.

    Overview

    The vCon library provides a complete Python implementation of the latest vCon specification for representing virtual conversations. It supports all features including parties, dialogs, attachments, analysis, digital signatures, extensibility, and advanced extensions for lawful basis management and standardized transcription formats.

    Installation

    For image processing support:

    Core Classes

    Vcon Class

    The main class for working with vCon objects.

    Constructor

    Parameters:

    • vcon_dict (Dict[str, Any], optional): Dictionary representing a vCon. Defaults to empty dict.

    • property_handling (str): How to handle non-standard properties:

      • "default": Keep non-standard properties (default)

    Class Methods

    build_new() -> Vcon

    Create a new vCon object with default values.

    build_from_json(json_str: str, property_handling: str = "default") -> Vcon

    Create a vCon object from JSON string.

    load(file_path_or_url: str, property_handling: str = "default") -> Vcon

    Load a vCon from file or URL.

    load_from_file(file_path: str, property_handling: str = "default") -> Vcon

    Load a vCon from a local file.

    load_from_url(url: str, property_handling: str = "default") -> Vcon

    Load a vCon from a URL.

    validate_file(file_path: str) -> Tuple[bool, List[str]]

    Validate a vCon file.

    validate_json(json_str: str) -> Tuple[bool, List[str]]

    Validate a vCon JSON string.

    generate_key_pair() -> Tuple[rsa.RSAPrivateKey, rsa.RSAPublicKey]

    Generate RSA key pair for digital signatures.

    uuid8_domain_name(domain_name: str) -> str

    Generate UUID8 with domain name.

    uuid8_time(custom_c_62_bits: int) -> str

    Generate UUID8 with custom time bits.

    Instance Methods

    Party Management

    add_party(party: Party) -> None

    Add a party to the vCon.

    find_party_index(by: str, val: str) -> Optional[int]

    Find party index by field value.

    Dialog Management

    add_dialog(dialog: Dialog) -> None

    Add a dialog to the vCon.

    find_dialog(by: str, val: str) -> Optional[Dialog]

    Find dialog by field value.

    find_dialogs_by_type(type: str) -> List[Dict[str, Any]]

    Find all dialogs of a specific type.

    add_transfer_dialog(start: Union[datetime, str], transfer_data: Dict[str, Any], parties: List[int]) -> None

    Add a transfer dialog.

    add_incomplete_dialog(start: Union[datetime, str], disposition: str, parties: List[int]) -> None

    Add an incomplete dialog.

    Attachment Management

    add_attachment(type: str, body: Any, encoding: str = "none") -> Attachment

    Add an attachment to the vCon.

    add_image(image_path: str, type: str = "image") -> Attachment

    Add an image attachment from file.

    find_attachment_by_type(type: str) -> Optional[Dict[str, Any]]

    Find attachment by type.

    Analysis Management

    add_analysis(type: str, dialog: Union[int, List[int]], vendor: str, body: Any, encoding: str = "none") -> None

    Add analysis data to the vCon.

    find_analysis_by_type(type: str) -> Optional[Dict[str, Any]]

    Find analysis by type.

    Tag Management

    add_tag(tag_name: str, tag_value: str) -> None

    Add a tag to the vCon.

    get_tag(tag_name: str) -> Optional[str]

    Get a tag value.

    Extension Management

    add_extension(extension: str) -> None

    Add an extension to the vCon.

    get_extensions() -> List[str]

    Get list of extensions.

    remove_extension(extension: str) -> None

    Remove an extension.

    add_must_support(extension: str) -> None

    Add a must-support extension.

    get_must_support() -> List[str]

    Get list of must-support extensions.

    remove_must_support(extension: str) -> None

    Remove a must-support extension.

    Extension-Specific Methods

    add_lawful_basis_attachment(lawful_basis: str, expiration: str, purpose_grants: list, party_index: Optional[int] = None, dialog_index: Optional[int] = None, **kwargs) -> None

    Add a lawful basis attachment to the vCon.

    add_wtf_transcription_attachment(transcript: Dict[str, Any], segments: List[Dict[str, Any]], metadata: Dict[str, Any], party_index: Optional[int] = None, dialog_index: Optional[int] = None, **kwargs) -> None

    Add a WTF transcription attachment to the vCon.

    find_lawful_basis_attachments(party_index: Optional[int] = None) -> List[Dict[str, Any]]

    Find lawful basis attachments in the vCon.

    find_wtf_attachments(party_index: Optional[int] = None) -> List[Dict[str, Any]]

    Find WTF transcription attachments in the vCon.

    check_lawful_basis_permission(purpose: str, party_index: Optional[int] = None) -> bool

    Check if permission is granted for a specific purpose.

    validate_extensions() -> Dict[str, Any]

    Validate all extensions in the vCon.

    process_extensions() -> Dict[str, Any]

    Process all extensions in the vCon.

    Security

    sign(private_key: Union[rsa.RSAPrivateKey, bytes]) -> None

    Sign the vCon with a private key.

    verify(public_key: Union[rsa.RSAPublicKey, bytes]) -> bool

    Verify the vCon signature.

    Validation

    is_valid() -> Tuple[bool, List[str]]

    Validate the vCon object.

    Serialization

    to_json() -> str

    Convert vCon to JSON string.

    to_dict() -> Dict[str, Any]

    Convert vCon to dictionary.

    dumps() -> str

    Alias for to_json().

    save_to_file(file_path: str) -> None

    Save vCon to file.

    HTTP Operations

    post_to_url(url: str, headers: Optional[Dict[str, str]] = None) -> requests.Response

    Post vCon to URL.

    Timestamp Management

    set_created_at(created_at: Union[str, datetime]) -> None

    Set the creation timestamp.

    set_updated_at(timestamp: Union[str, datetime]) -> None

    Set the update timestamp.

    Properties

    uuid -> str

    Get the vCon UUID.

    vcon -> Optional[str]

    Get the vCon version (optional field).

    subject -> Optional[str]

    Get the vCon subject.

    created_at

    Get the creation timestamp.

    updated_at

    Get the update timestamp.

    redacted

    Get the redacted flag.

    appended

    Get the appended flag.

    group

    Get the group information.

    meta

    Get the metadata.

    parties -> List[Party]

    Get list of parties.

    dialog -> List[Dict[str, Any]]

    Get list of dialogs.

    attachments -> List[Dict[str, Any]]

    Get list of attachments.

    analysis -> List[Dict[str, Any]]

    Get list of analysis data.

    tags -> Optional[Dict[str, Any]]

    Get all tags.

    Party Class

    Represents a participant in a vCon conversation.

    Constructor

    Parameters:

    • tel (str, optional): Telephone number

    • stir (str, optional): STIR identifier

    • mailto (str, optional): Email address

    Methods

    to_dict() -> Dict[str, Any]

    Convert Party to dictionary.

    Dialog Class

    Represents a dialog segment in a vCon conversation.

    Constructor

    Parameters:

    • type (str): Dialog type ("text", "recording", "transfer", "incomplete", "audio", "video")

    • start (Union[datetime, str]): Start time

    • parties (List[int]): List of party indices

    Methods

    to_dict() -> Dict[str, Any]

    Convert Dialog to dictionary.

    add_external_data(url: str, filename: str, mimetype: str) -> None

    Add external data to dialog.

    add_inline_data(body: str, filename: str, mimetype: str) -> None

    Add inline data to dialog.

    is_external_data() -> bool

    Check if dialog has external data.

    is_inline_data() -> bool

    Check if dialog has inline data.

    is_text() -> bool

    Check if dialog is text type.

    is_recording() -> bool

    Check if dialog is recording type.

    is_transfer() -> bool

    Check if dialog is transfer type.

    is_incomplete() -> bool

    Check if dialog is incomplete type.

    is_audio() -> bool

    Check if dialog has audio content.

    is_video(content_type: Optional[str] = None) -> bool

    Check if dialog has video content.

    is_email() -> bool

    Check if dialog is email type.

    is_image() -> bool

    Check if dialog has image content.

    is_pdf() -> bool

    Check if dialog has PDF content.

    add_video_data(video_data, filename: Optional[str] = None, mimetype: Optional[str] = None, inline: bool = True, metadata: Optional[dict] = None) -> None

    Add video data to dialog.

    extract_video_metadata(video_path: Optional[str] = None) -> dict

    Extract video metadata using FFmpeg.

    generate_thumbnail(timestamp: float = 0.0, width: int = 320, height: int = 240, quality: int = 90) -> bytes

    Generate video thumbnail.

    add_streaming_video_reference(reference_id: str, mimetype: str, metadata: Optional[dict] = None) -> None

    Add streaming video reference.

    add_video_with_optimal_storage(video_data, filename: str, mimetype: Optional[str] = None, size_threshold_mb: int = 10) -> None

    Add video with optimal storage method.

    transcode_video(target_format: str, codec: Optional[str] = None, bit_rate: Optional[int] = None, width: Optional[int] = None, height: Optional[int] = None) -> None

    Transcode video to different format.

    add_image_data(image_path: str, mimetype: Optional[str] = None) -> None

    Add image data from file.

    extract_image_metadata(image_data: bytes, mimetype: str) -> None

    Extract image metadata.

    generate_thumbnail(max_size: Tuple[int, int] = (200, 200)) -> Optional[str]

    Generate image thumbnail.

    is_external_data_changed() -> bool

    Check if external data has changed.

    to_inline_data() -> None

    Convert external data to inline data.

    set_session_id(session_id: str) -> None

    Set session identifier.

    get_session_id() -> Optional[str]

    Get session identifier.

    set_content_hash(content_hash: str) -> None

    Set content hash.

    get_content_hash() -> Optional[str]

    Get content hash.

    calculate_content_hash(algorithm: str = "sha256") -> str

    Calculate content hash.

    verify_content_hash(expected_hash: str, algorithm: str = "sha256") -> bool

    Verify content hash.

    CivicAddress Class

    Represents civic address information according to GEOPRIV specification.

    Constructor

    Parameters:

    • country (str, optional): Country code (ISO 3166-1 alpha-2)

    • a1 (str, optional): Administrative area 1 (state/province)

    • a2 (str, optional): Administrative area 2 (county/municipality)

    Methods

    to_dict() -> Dict[str, Optional[str]]

    Convert CivicAddress to dictionary.

    PartyHistory Class

    Represents party history events in a vCon dialog.

    Constructor

    Parameters:

    • party (int): Index of the party

    • event (str): Event type ("join", "drop", "hold", "unhold", "mute", "unmute")

    • time (datetime): Time of the event

    Methods

    to_dict() -> Dict[str, Any]

    Convert PartyHistory to dictionary.

    Attachment Class

    Represents an attachment in a vCon.

    Constructor

    Parameters:

    • type (str): Type of attachment

    • body (Any): Content of attachment

    • encoding (str): Encoding format ("base64", "base64url", "none")

    Methods

    to_dict() -> Dict[str, Any]

    Convert Attachment to dictionary.

    from_image(image_path: str, type: str = "image") -> 'Attachment'

    Create attachment from image file.

    Extensions

    The vCon library includes a comprehensive extension framework that allows for standardized implementation of additional functionality. Two major extensions are currently implemented: the Lawful Basis extension for privacy compliance and the WTF (World Transcription Format) extension for standardized transcription data.

    Extension Framework

    The extension framework provides a standardized way to add new functionality to vCon objects while maintaining compatibility and validation.

    Core Extension Classes

    ExtensionType

    Enumeration of extension types:

    • COMPATIBLE: Safe to ignore, no breaking changes

    • INCOMPATIBLE: Must be supported, breaking changes

    • EXPERIMENTAL: Development/testing only

    ExtensionValidator

    Abstract base class for extension validation logic.

    ExtensionProcessor

    Abstract base class for extension processing logic.

    ExtensionRegistry

    Central registry for managing extensions.

    Lawful Basis Extension

    The Lawful Basis extension provides comprehensive support for privacy compliance and consent management according to GDPR and other privacy regulations.

    Key Features

    • Multiple Lawful Basis Types: consent, contract, legal_obligation, vital_interests, public_task, legitimate_interests

    • Purpose-Specific Permissions: Granular permission grants with conditions

    • Cryptographic Proof Mechanisms: Verbal confirmation, signed documents, cryptographic signatures, external systems

    • Temporal Validity: Expiration dates and status intervals

    Core Classes

    LawfulBasisAttachment

    Main class representing a lawful basis attachment.

    PurposeGrant

    Represents a purpose-specific permission grant.

    ContentHash

    Represents content integrity information.

    ProofMechanism

    Represents a proof mechanism for lawful basis.

    Validation and Processing

    LawfulBasisValidator

    Validates lawful basis attachments and extension usage.

    LawfulBasisProcessor

    Processes lawful basis attachments and evaluates permissions.

    Registry Integration

    SCITTRegistryClient

    Client for SCITT (Supply Chain Integrity, Transparency, and Trust) registries.

    WTF Extension

    The WTF (World Transcription Format) extension provides standardized representation of speech-to-text transcription data from multiple providers.

    Key Features

    • Multi-Provider Support: Whisper, Deepgram, AssemblyAI, Google, Amazon, Azure, and more

    • Standardized Format: Hierarchical structure with transcripts, segments, words, and speakers

    • Quality Metrics: Audio quality assessment and confidence scoring

    • Export Capabilities: SRT and WebVTT subtitle formats

    Core Classes

    WTFAttachment

    Main class representing a WTF transcription attachment.

    Transcript

    Represents high-level transcript information.

    Segment

    Represents a logical chunk of transcribed content.

    Word

    Represents a single word in the transcription.

    Speaker

    Represents speaker information for diarization.

    Quality

    Represents quality metrics for the transcription.

    Provider Adapters

    WhisperAdapter

    Converts Whisper transcription data to WTF format.

    DeepgramAdapter

    Converts Deepgram transcription data to WTF format.

    AssemblyAIAdapter

    Converts AssemblyAI transcription data to WTF format.

    Export Capabilities

    SRT Export

    Export transcription to SRT subtitle format.

    WebVTT Export

    Export transcription to WebVTT format.

    Analysis Tools

    Keyword Extraction

    Extract keywords from high-confidence words.

    Low Confidence Detection

    Find segments with confidence below threshold.

    Speaking Time Calculation

    Calculate speaking time for each speaker.

    Validation and Processing

    WTFValidator

    Validates WTF transcription attachments.

    WTFProcessor

    Processes WTF transcription attachments and provides analysis.

    Constants

    Property Handling Modes

    Dialog Types

    Disposition Values

    Party History Events

    Attachment Encodings

    Supported MIME Types

    Extension Types

    Lawful Basis Types

    Proof Types

    Hash Algorithms

    Canonicalization Methods

    WTF Provider Adapters

    Examples

    Basic vCon Creation

    Loading and Validation

    Digital Signatures

    Extensions and Must-Support

    Analysis and Attachments

    Video Content

    Civic Address

    Party History

    HTTP Operations

    Property Handling

    Lawful Basis Extension

    WTF Extension

    Extension Validation and Processing

    Provider Data Conversion

    Complete Extension Workflow

    This API reference covers all the main functionality of the vCon library, including the new extension framework. For more detailed examples and use cases, see the Quickstart Guide and the samples directory.

    "strict": Remove non-standard properties

  • "meta": Move non-standard properties to meta object

  • name (str, optional): Display name
  • validation (str, optional): Validation information

  • gmlpos (str, optional): GML position coordinates

  • civicaddress (CivicAddress, optional): Civic address information

  • uuid (str, optional): Unique identifier

  • role (str, optional): Role in conversation (e.g., "caller", "agent")

  • contact_list (str, optional): Contact list reference

  • meta (dict, optional): Additional metadata

  • sip (str, optional): SIP URI for VoIP communication

  • did (str, optional): Decentralized Identifier

  • jCard (dict, optional): vCard format contact information

  • timezone (str, optional): Party's timezone

  • originator (int, optional): Originator party index
  • mimetype (str, optional): MIME type of content

  • filename (str, optional): Filename

  • body (str, optional): Content body

  • encoding (str, optional): Content encoding

  • url (str, optional): External URL

  • alg (str, optional): Signature algorithm

  • signature (str, optional): Content signature

  • disposition (str, optional): Disposition for incomplete dialogs

  • party_history (List[PartyHistory], optional): Party event history

  • transferee (int, optional): Transferee party index

  • transferor (int, optional): Transferor party index

  • transfer_target (int, optional): Transfer target party index

  • original (int, optional): Original dialog index

  • consultation (int, optional): Consultation dialog index

  • target_dialog (int, optional): Target dialog index

  • campaign (str, optional): Campaign identifier

  • interaction (str, optional): Interaction identifier

  • skill (str, optional): Skill identifier

  • duration (float, optional): Dialog duration

  • meta (dict, optional): Additional metadata

  • metadata (Dict[str, Any], optional): Structured metadata

  • transfer (Dict[str, Any], optional): Transfer-specific information

  • signaling (Dict[str, Any], optional): Signaling information

  • resolution (str, optional): Video resolution (e.g., "1920x1080")

  • frame_rate (float, optional): Video frame rate

  • codec (str, optional): Video codec

  • bitrate (int, optional): Video bitrate

  • thumbnail (str, optional): Base64-encoded thumbnail

  • session_id (str, optional): Session identifier

  • content_hash (str, optional): Content hash for external files

  • application (str, optional): Application identifier

  • message_id (str, optional): Message identifier

  • a3 (str, optional): Administrative area 3 (city/town)
  • a4 (str, optional): Administrative area 4 (neighborhood/district)

  • a5 (str, optional): Administrative area 5 (postal code)

  • a6 (str, optional): Administrative area 6 (building/floor)

  • prd (str, optional): Premier (department/suite number)

  • pod (str, optional): Post office box identifier

  • sts (str, optional): Street name

  • hno (str, optional): House number

  • hns (str, optional): House name

  • lmk (str, optional): Landmark name

  • loc (str, optional): Location name

  • flr (str, optional): Floor

  • nam (str, optional): Name of location

  • pc (str, optional): Postal code

  • Content Integrity: Hash validation and canonicalization

  • External Registry Integration: SCITT (Supply Chain Integrity, Transparency, and Trust) support

  • Provider Adapters: Automatic conversion from provider-specific formats

  • Analysis Tools: Keyword extraction, confidence analysis, and transcription comparison

  • pip install vcon
    pip install vcon[image]
    Vcon(vcon_dict: Dict[str, Any] = None, property_handling: str = "default")
    vcon = Vcon.build_new()
    vcon = Vcon.build_from_json('{"uuid": "123", "created_at": "2024-01-01T00:00:00Z"}')
    # From file
    vcon = Vcon.load("conversation.vcon.json")
    
    # From URL
    vcon = Vcon.load("https://example.com/conversation.vcon.json")
    is_valid, errors = Vcon.validate_file("conversation.vcon.json")
    private_key, public_key = Vcon.generate_key_pair()
    party = Party(tel="+1234567890", name="Alice", role="caller")
    vcon.add_party(party)
    index = vcon.find_party_index("tel", "+1234567890")
    dialog = Dialog(type="text", start=datetime.now(), parties=[0, 1], body="Hello")
    vcon.add_dialog(dialog)
    dialog = vcon.find_dialog("type", "text")
    text_dialogs = vcon.find_dialogs_by_type("text")
    transfer_data = {
        "transferee": 0,
        "transferor": 1,
        "transfer_target": 2
    }
    vcon.add_transfer_dialog(datetime.now(), transfer_data, [0, 1, 2])
    vcon.add_incomplete_dialog(datetime.now(), "no-answer", [0])
    attachment = vcon.add_attachment("transcript", "Full conversation...", "none")
    attachment = vcon.add_image("screenshot.png", "screenshot")
    transcript = vcon.find_attachment_by_type("transcript")
    vcon.add_analysis(
        type="sentiment",
        dialog=[0, 1],
        vendor="SentimentAnalyzer",
        body={"sentiment": "positive", "confidence": 0.85},
        encoding="json"
    )
    sentiment = vcon.find_analysis_by_type("sentiment")
    vcon.add_tag("customer_id", "12345")
    customer_id = vcon.get_tag("customer_id")
    vcon.add_extension("video")
    extensions = vcon.get_extensions()
    vcon.remove_extension("video")
    vcon.add_must_support("encryption")
    must_support = vcon.get_must_support()
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration="2026-01-01T00:00:00Z",
        purpose_grants=[
            {"purpose": "recording", "granted": True, "granted_at": "2025-01-01T00:00:00Z"}
        ],
        party_index=0
    )
    vcon.add_wtf_transcription_attachment(
        transcript={"text": "Hello world", "language": "en", "duration": 2.0, "confidence": 0.95},
        segments=[{"id": 0, "start": 0.0, "end": 2.0, "text": "Hello world", "confidence": 0.95}],
        metadata={"created_at": "2025-01-01T00:00:00Z", "provider": "whisper", "model": "whisper-1"}
    )
    attachments = vcon.find_lawful_basis_attachments(party_index=0)
    attachments = vcon.find_wtf_attachments(party_index=0)
    has_permission = vcon.check_lawful_basis_permission("recording", party_index=0)
    results = vcon.validate_extensions()
    results = vcon.process_extensions()
    vcon.sign(private_key)
    is_valid = vcon.verify(public_key)
    is_valid, errors = vcon.is_valid()
    json_str = vcon.to_json()
    vcon_dict = vcon.to_dict()
    vcon.save_to_file("conversation.vcon.json")
    response = vcon.post_to_url("https://api.example.com/vcon", headers={"Authorization": "Bearer token"})
    Party(
        tel: Optional[str] = None,
        stir: Optional[str] = None,
        mailto: Optional[str] = None,
        name: Optional[str] = None,
        validation: Optional[str] = None,
        gmlpos: Optional[str] = None,
        civicaddress: Optional[CivicAddress] = None,
        uuid: Optional[str] = None,
        role: Optional[str] = None,
        contact_list: Optional[str] = None,
        meta: Optional[dict] = None,
        sip: Optional[str] = None,
        did: Optional[str] = None,
        jCard: Optional[dict] = None,
        timezone: Optional[str] = None,
        **kwargs
    )
    party_dict = party.to_dict()
    Dialog(
        type: str,
        start: Union[datetime, str],
        parties: List[int],
        originator: Optional[int] = None,
        mimetype: Optional[str] = None,
        filename: Optional[str] = None,
        body: Optional[str] = None,
        encoding: Optional[str] = None,
        url: Optional[str] = None,
        alg: Optional[str] = None,
        signature: Optional[str] = None,
        disposition: Optional[str] = None,
        party_history: Optional[List[PartyHistory]] = None,
        transferee: Optional[int] = None,
        transferor: Optional[int] = None,
        transfer_target: Optional[int] = None,
        original: Optional[int] = None,
        consultation: Optional[int] = None,
        target_dialog: Optional[int] = None,
        campaign: Optional[str] = None,
        interaction: Optional[str] = None,
        skill: Optional[str] = None,
        duration: Optional[float] = None,
        meta: Optional[dict] = None,
        metadata: Optional[Dict[str, Any]] = None,
        transfer: Optional[Dict[str, Any]] = None,
        signaling: Optional[Dict[str, Any]] = None,
        resolution: Optional[str] = None,
        frame_rate: Optional[float] = None,
        codec: Optional[str] = None,
        bitrate: Optional[int] = None,
        thumbnail: Optional[str] = None,
        session_id: Optional[str] = None,
        content_hash: Optional[str] = None,
        application: Optional[str] = None,
        message_id: Optional[str] = None,
        **kwargs
    )
    CivicAddress(
        country: Optional[str] = None,
        a1: Optional[str] = None,
        a2: Optional[str] = None,
        a3: Optional[str] = None,
        a4: Optional[str] = None,
        a5: Optional[str] = None,
        a6: Optional[str] = None,
        prd: Optional[str] = None,
        pod: Optional[str] = None,
        sts: Optional[str] = None,
        hno: Optional[str] = None,
        hns: Optional[str] = None,
        lmk: Optional[str] = None,
        loc: Optional[str] = None,
        flr: Optional[str] = None,
        nam: Optional[str] = None,
        pc: Optional[str] = None
    )
    PartyHistory(party: int, event: str, time: datetime)
    Attachment(type: str, body: Any, encoding: str = "none")
    from vcon.extensions import get_extension_registry
    
    # Get the global registry
    registry = get_extension_registry()
    
    # List all registered extensions
    extensions = registry.list_extensions()
    from vcon.extensions.lawful_basis import LawfulBasisAttachment, LawfulBasisType, PurposeGrant
    from datetime import datetime, timezone, timedelta
    
    # Create purpose grants
    purpose_grants = [
        PurposeGrant(
            purpose="recording",
            granted=True,
            granted_at=datetime.now(timezone.utc).isoformat()
        ),
        PurposeGrant(
            purpose="analysis",
            granted=True,
            granted_at=datetime.now(timezone.utc).isoformat(),
            conditions=["anonymized_data_only"]
        )
    ]
    
    # Create lawful basis attachment
    attachment = LawfulBasisAttachment(
        lawful_basis=LawfulBasisType.CONSENT,
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=purpose_grants
    )
    grant = PurposeGrant(
        purpose="recording",
        granted=True,
        granted_at=datetime.now(timezone.utc).isoformat(),
        conditions=["anonymized_data_only"]
    )
    from vcon.extensions.lawful_basis import ContentHash, HashAlgorithm, CanonicalizationMethod
    
    content_hash = ContentHash(
        algorithm=HashAlgorithm.SHA_256,
        canonicalization=CanonicalizationMethod.JCS,
        value="computed_hash_value"
    )
    from vcon.extensions.lawful_basis import ProofMechanism, ProofType
    
    proof = ProofMechanism(
        proof_type=ProofType.VERBAL_CONFIRMATION,
        timestamp=datetime.now(timezone.utc).isoformat(),
        proof_data={
            "dialog_reference": 0,
            "confirmation_text": "I consent to recording"
        }
    )
    from vcon.extensions.lawful_basis import LawfulBasisValidator
    
    validator = LawfulBasisValidator()
    result = validator.validate_attachment(attachment_dict)
    from vcon.extensions.lawful_basis import LawfulBasisProcessor
    
    processor = LawfulBasisProcessor()
    result = processor.check_permission(vcon_dict, "recording", party_index=0)
    from vcon.extensions.lawful_basis import SCITTRegistryClient
    
    client = SCITTRegistryClient("https://registry.example.com", auth_token="token")
    receipt_id = client.submit_attestation(lawful_basis_attachment)
    from vcon.extensions.wtf import WTFAttachment, Transcript, Segment, Metadata
    from datetime import datetime, timezone
    
    # Create transcript
    transcript = Transcript(
        text="Hello world",
        language="en",
        duration=2.0,
        confidence=0.95
    )
    
    # Create segments
    segments = [
        Segment(
            id=0,
            start=0.0,
            end=2.0,
            text="Hello world",
            confidence=0.95
        )
    ]
    
    # Create metadata
    metadata = Metadata(
        created_at=datetime.now(timezone.utc).isoformat(),
        processed_at=datetime.now(timezone.utc).isoformat(),
        provider="whisper",
        model="whisper-1"
    )
    
    # Create WTF attachment
    attachment = WTFAttachment(
        transcript=transcript,
        segments=segments,
        metadata=metadata
    )
    transcript = Transcript(
        text="Hello world",
        language="en",
        duration=2.0,
        confidence=0.95
    )
    segment = Segment(
        id=0,
        start=0.0,
        end=2.0,
        text="Hello world",
        confidence=0.95,
        speaker=0
    )
    word = Word(
        id=0,
        start=0.0,
        end=1.0,
        text="Hello",
        confidence=0.95,
        speaker=0
    )
    speaker = Speaker(
        id=0,
        label="Speaker 1",
        segments=[0, 1, 2],
        total_time=10.5,
        confidence=0.9
    )
    quality = Quality(
        audio_quality="high",
        background_noise=0.1,
        multiple_speakers=True,
        overlapping_speech=False,
        silence_ratio=0.2,
        average_confidence=0.95,
        low_confidence_words=5,
        processing_warnings=[]
    )
    from vcon.extensions.wtf import WhisperAdapter
    
    adapter = WhisperAdapter()
    wtf_attachment = adapter.convert(whisper_data)
    from vcon.extensions.wtf import DeepgramAdapter
    
    adapter = DeepgramAdapter()
    wtf_attachment = adapter.convert(deepgram_data)
    from vcon.extensions.wtf import AssemblyAIAdapter
    
    adapter = AssemblyAIAdapter()
    wtf_attachment = adapter.convert(assemblyai_data)
    srt_content = attachment.export_to_srt()
    vtt_content = attachment.export_to_vtt()
    keywords = attachment.extract_keywords(min_confidence=0.8)
    low_confidence_segments = attachment.find_low_confidence_segments(threshold=0.5)
    speaking_times = attachment.get_speaking_time()
    from vcon.extensions.wtf import WTFValidator
    
    validator = WTFValidator()
    result = validator.validate_attachment(attachment_dict)
    from vcon.extensions.wtf import WTFProcessor
    
    processor = WTFProcessor()
    analysis = processor.analyze_transcription(attachment)
    PROPERTY_HANDLING_DEFAULT = "default"  # Keep non-standard properties
    PROPERTY_HANDLING_STRICT = "strict"    # Remove non-standard properties
    PROPERTY_HANDLING_META = "meta"        # Move non-standard properties to meta
    VALID_TYPES = ["recording", "text", "transfer", "incomplete", "audio", "video"]
    VALID_DISPOSITIONS = [
        "no-answer", "congestion", "failed", "busy", 
        "hung-up", "voicemail-no-message"
    ]
    VALID_EVENTS = ["join", "drop", "hold", "unhold", "mute", "unmute"]
    VALID_ENCODINGS = ["base64", "base64url", "none"]
    MIME_TYPES = [
        "text/plain",
        "audio/x-wav", "audio/wav", "audio/wave", "audio/mpeg", "audio/mp3",
        "audio/x-mp3", "audio/x-mp4", "audio/ogg", "audio/webm", "audio/x-m4a", "audio/aac",
        "video/x-mp4", "video/ogg", "video/mp4", "video/quicktime", "video/webm",
        "video/x-msvideo", "video/x-matroska", "video/mpeg", "video/x-flv", "video/3gpp", "video/x-m4v",
        "multipart/mixed", "message/rfc822",
        "image/jpeg", "image/tiff", "application/pdf", "application/json"
    ]
    from vcon.extensions.base import ExtensionType
    
    ExtensionType.COMPATIBLE     # Safe to ignore, no breaking changes
    ExtensionType.INCOMPATIBLE   # Must be supported, breaking changes
    ExtensionType.EXPERIMENTAL   # Development/testing only
    from vcon.extensions.lawful_basis import LawfulBasisType
    
    LawfulBasisType.CONSENT              # Explicit consent
    LawfulBasisType.CONTRACT             # Contractual necessity
    LawfulBasisType.LEGAL_OBLIGATION     # Legal obligation
    LawfulBasisType.VITAL_INTERESTS      # Vital interests
    LawfulBasisType.PUBLIC_TASK          # Public task
    LawfulBasisType.LEGITIMATE_INTERESTS # Legitimate interests
    from vcon.extensions.lawful_basis import ProofType
    
    ProofType.VERBAL_CONFIRMATION        # Verbal confirmation
    ProofType.SIGNED_DOCUMENT           # Signed document
    ProofType.CRYPTOGRAPHIC_SIGNATURE   # Cryptographic signature
    ProofType.EXTERNAL_SYSTEM           # External system attestation
    from vcon.extensions.lawful_basis import HashAlgorithm
    
    HashAlgorithm.SHA_256    # SHA-256
    HashAlgorithm.SHA_384    # SHA-384
    HashAlgorithm.SHA_512    # SHA-512
    from vcon.extensions.lawful_basis import CanonicalizationMethod
    
    CanonicalizationMethod.JCS    # JSON Canonicalization Scheme
    from vcon.extensions.wtf import (
        WhisperAdapter,      # OpenAI Whisper
        DeepgramAdapter,     # Deepgram
        AssemblyAIAdapter,   # AssemblyAI
        ProviderAdapter      # Base adapter class
    )
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime
    
    # Create new vCon
    vcon = Vcon.build_new()
    
    # Add parties
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add dialog
    dialog = Dialog(
        type="text",
        start=datetime.now(),
        parties=[0, 1],
        originator=0,
        body="Hello, I need help with my account."
    )
    vcon.add_dialog(dialog)
    
    # Save to file
    vcon.save_to_file("conversation.vcon.json")
    # Load vCon
    vcon = Vcon.load("conversation.vcon.json")
    
    # Validate
    is_valid, errors = vcon.is_valid()
    if not is_valid:
        print("Validation errors:", errors)
    
    # Access data
    print(f"UUID: {vcon.uuid}")
    print(f"Parties: {len(vcon.parties)}")
    print(f"Dialogs: {len(vcon.dialog)}")
    # Generate key pair
    private_key, public_key = Vcon.generate_key_pair()
    
    # Sign vCon
    vcon.sign(private_key)
    
    # Verify signature
    is_valid = vcon.verify(public_key)
    print(f"Signature valid: {is_valid}")
    # Add extensions
    vcon.add_extension("video")
    vcon.add_extension("encryption")
    
    # Add must-support
    vcon.add_must_support("encryption")
    
    print(f"Extensions: {vcon.get_extensions()}")
    print(f"Must support: {vcon.get_must_support()}")
    # Add analysis
    vcon.add_analysis(
        type="sentiment",
        dialog=[0, 1],
        vendor="SentimentAnalyzer",
        body={"sentiment": "positive", "confidence": 0.85},
        encoding="json"
    )
    
    # Add attachment
    vcon.add_attachment(
        type="transcript",
        body="Full conversation transcript...",
        encoding="none"
    )
    
    # Add tags
    vcon.add_tag("customer_id", "12345")
    vcon.add_tag("priority", "high")
    # Add video dialog
    video_dialog = Dialog(
        type="video",
        start=datetime.now(),
        parties=[0, 1],
        mimetype="video/mp4",
        resolution="1920x1080",
        frame_rate=30.0,
        codec="H.264"
    )
    
    # Add video data
    video_dialog.add_video_data(
        video_data=binary_video_data,
        filename="recording.mp4",
        mimetype="video/mp4",
        inline=True
    )
    
    # Extract metadata
    metadata = video_dialog.extract_video_metadata()
    
    # Generate thumbnail
    thumbnail = video_dialog.generate_thumbnail(timestamp=10.0)
    
    vcon.add_dialog(video_dialog)
    from vcon.civic_address import CivicAddress
    
    # Create civic address
    address = CivicAddress(
        country="US",
        a1="CA",
        a3="San Francisco",
        sts="Market Street",
        hno="123",
        pc="94102"
    )
    
    # Add to party
    party = Party(
        name="Jane",
        tel="+1555123456",
        civicaddress=address
    )
    from vcon.party import PartyHistory
    
    # Create party history
    history = [
        PartyHistory(0, "join", datetime.now()),
        PartyHistory(1, "join", datetime.now()),
        PartyHistory(0, "hold", datetime.now()),
        PartyHistory(0, "unhold", datetime.now()),
        PartyHistory(1, "drop", datetime.now())
    ]
    
    # Add to dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(),
        parties=[0, 1],
        party_history=history
    )
    # Post vCon to server
    response = vcon.post_to_url(
        "https://api.example.com/vcon",
        headers={
            "Authorization": "Bearer your-token",
            "Content-Type": "application/json"
        }
    )
    
    if response.status_code == 200:
        print("Successfully posted vCon")
    else:
        print(f"Error: {response.status_code}")
    # Strict mode - remove non-standard properties
    vcon = Vcon.load("file.json", property_handling="strict")
    
    # Meta mode - move non-standard properties to meta
    vcon = Vcon.load("file.json", property_handling="meta")
    
    # Default mode - keep all properties
    vcon = Vcon.load("file.json", property_handling="default")
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime, timezone, timedelta
    
    # Create vCon with parties and dialog
    vcon = Vcon.build_new()
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(timezone.utc),
        parties=[0, 1],
        mimetype="audio/mp3"
    )
    vcon.add_dialog(dialog)
    
    # Add lawful basis attachment
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "analysis",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat(),
                "conditions": ["anonymized_data_only"]
            }
        ],
        party_index=0,
        dialog_index=0
    )
    
    # Check permissions
    recording_permission = vcon.check_lawful_basis_permission("recording", party_index=0)
    marketing_permission = vcon.check_lawful_basis_permission("marketing", party_index=0)
    
    print(f"Recording permission: {recording_permission}")
    print(f"Marketing permission: {marketing_permission}")
    
    # Find lawful basis attachments
    attachments = vcon.find_lawful_basis_attachments(party_index=0)
    print(f"Found {len(attachments)} lawful basis attachments")
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime, timezone
    
    # Create vCon with parties and dialog
    vcon = Vcon.build_new()
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(timezone.utc),
        parties=[0, 1],
        mimetype="audio/mp3"
    )
    vcon.add_dialog(dialog)
    
    # Add WTF transcription attachment
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, this is a test transcription.",
            "language": "en",
            "duration": 3.5,
            "confidence": 0.95
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 1.5,
                "text": "Hello, this is",
                "confidence": 0.95,
                "speaker": 0
            },
            {
                "id": 1,
                "start": 1.5,
                "end": 3.5,
                "text": "a test transcription.",
                "confidence": 0.94,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1",
            "audio_quality": "high",
            "background_noise": 0.1
        },
        party_index=0,
        dialog_index=0
    )
    
    # Find WTF attachments
    attachments = vcon.find_wtf_attachments(party_index=0)
    print(f"Found {len(attachments)} WTF attachments")
    
    # Export to SRT format
    if attachments:
        from vcon.extensions.wtf import WTFAttachment
        wtf_attachment = WTFAttachment.from_dict(attachments[0]["body"])
        srt_content = wtf_attachment.export_to_srt()
        print("SRT Export:")
        print(srt_content)
    # Validate all extensions
    validation_results = vcon.validate_extensions()
    print("Extension validation results:")
    for extension, result in validation_results.items():
        if extension != "attachments":
            status = "✓ Valid" if result["is_valid"] else "✗ Invalid"
            print(f"  {extension}: {status}")
            if result["errors"]:
                for error in result["errors"]:
                    print(f"    Error: {error}")
            if result["warnings"]:
                for warning in result["warnings"]:
                    print(f"    Warning: {warning}")
    
    # Process all extensions
    processing_results = vcon.process_extensions()
    print("Extension processing completed")
    from vcon.extensions.wtf import WhisperAdapter, DeepgramAdapter
    
    # Convert Whisper data to WTF format
    whisper_data = {
        "text": "Hello world from Whisper",
        "segments": [
            {
                "start": 0.0,
                "end": 2.0,
                "text": "Hello world from Whisper"
            }
        ]
    }
    
    whisper_adapter = WhisperAdapter()
    wtf_attachment = whisper_adapter.convert(whisper_data)
    
    # Add to vCon
    vcon.add_wtf_transcription_attachment(
        transcript=wtf_attachment.transcript.to_dict(),
        segments=[segment.to_dict() for segment in wtf_attachment.segments],
        metadata=wtf_attachment.metadata.to_dict()
    )
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime, timezone, timedelta
    
    # Create comprehensive vCon with extensions
    vcon = Vcon.build_new()
    
    # Add parties
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(timezone.utc),
        parties=[0, 1],
        mimetype="audio/mp3"
    )
    vcon.add_dialog(dialog)
    
    # Add lawful basis for consent
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "transcription",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            }
        ],
        party_index=0
    )
    
    # Add transcription
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, I need help with my account.",
            "language": "en",
            "duration": 4.2,
            "confidence": 0.92
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 4.2,
                "text": "Hello, I need help with my account.",
                "confidence": 0.92,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1"
        },
        party_index=0,
        dialog_index=0
    )
    
    # Validate and process
    validation_results = vcon.validate_extensions()
    processing_results = vcon.process_extensions()
    
    # Check permissions
    can_record = vcon.check_lawful_basis_permission("recording", party_index=0)
    can_transcribe = vcon.check_lawful_basis_permission("transcription", party_index=0)
    
    print(f"Can record: {can_record}")
    print(f"Can transcribe: {can_transcribe}")
    
    # Save vCon
    vcon.save_to_file("conversation_with_extensions.vcon.json")
    print("Saved vCon with extensions")

    vCon Library Guide for LLMs

    Stick this in your robot's context window.

    This guide provides a comprehensive overview of the vCon (Virtual Conversation) Python library, designed specifically for Large Language Models (LLMs) that need to generate or modify code using this library.

    Overview

    The vCon library is a Python implementation of the vCon 0.3.0 specification for structuring, managing, and manipulating conversation data in a standardized format. It enables the creation, validation, and manipulation of digital representations of conversations with rich metadata, supporting all modern conversation features including multimedia content, security, and extensibility.

    Key Concepts

    • vCon Container: The primary object that holds all conversation data

    • Parties: Participants in a conversation (callers, agents, bots) with contact information

    • Dialogs: Individual messages or segments of the conversation (text, audio, video, etc.)

    • Attachments: Additional files or data associated with the conversation

    Installation

    Requirements

    • Python 3.12+

    • Core dependencies: authlib, uuid6, requests, pydash, python-dateutil

    • Optional: mutagen (audio metadata), ffmpeg (video processing), Pillow (image processing), PyPDF (PDF processing)

    Core Classes and Usage Patterns

    1. Vcon Class

    The main container for all conversation data.

    Creating a vCon

    Saving and Exporting

    Properties

    2. Party Class

    Represents a participant in the conversation.

    Party Attributes

    Core Contact Information:

    • tel: Telephone number (e.g., "+1234567890")

    • name: Display name (e.g., "Alice Smith")

    • role: Role in conversation ("caller", "agent", "bot", etc.)

    Advanced Contact Methods (vCon 0.3.0):

    • sip: SIP URI for VoIP communication (e.g., "sip:[email protected]")

    • did: Decentralized Identifier for blockchain-based identity

    • jCard: vCard format contact information (RFC 7095)

    Location and Validation:

    • civicaddress: Civic address using CivicAddress class (GEOPRIV format)

    • gmlpos: GML position coordinates

    • validation: Validation status

    Metadata:

    • uuid: Unique identifier for the party

    • contact_list: Reference to contact list

    • meta: Additional metadata dictionary

    3. Dialog Class

    Represents a message or segment in the conversation.

    Special Dialog Types

    Dialog Type Methods

    Dialog Types and MIME Types

    Valid Dialog Types:

    • "text": Text-based communication (chat, SMS, email)

    • "recording": Audio/video recording

    • "transfer": Call transfer operation

    Supported MIME Types:

    Text:

    • text/plain

    Audio:

    • audio/x-wav, audio/wav, audio/wave

    • audio/mpeg, audio/mp3, audio/x-mp3

    Video:

    • video/x-mp4, video/mp4, video/ogg

    • video/quicktime, video/webm

    • video/x-msvideo

    Other:

    • multipart/mixed

    • message/rfc822 (for email)

    • application/json (for signaling data)

    4. Working with Tags

    Tags are key-value pairs for simple metadata.

    5. Working with Attachments

    Attachments are arbitrary data associated with the conversation.

    6. Working with Analysis

    Analysis entries represent insights derived from dialog.

    7. Extensions and Must-Support (vCon 0.3.0)

    Extensions allow vCons to declare optional features they use, while must-support indicates required features.

    7.1. Lawful Basis Extension

    The Lawful Basis extension provides comprehensive support for privacy compliance and consent management according to GDPR and other privacy regulations.

    Key Features

    • Multiple Lawful Basis Types: consent, contract, legal_obligation, vital_interests, public_task, legitimate_interests

    • Purpose-Specific Permissions: Granular permission grants with conditions

    • Cryptographic Proof Mechanisms: Verbal confirmation, signed documents, cryptographic signatures, external systems

    • Temporal Validity: Expiration dates and status intervals

    Adding Lawful Basis Attachments

    Checking Permissions

    Finding Lawful Basis Attachments

    Advanced Lawful Basis Features

    7.2. WTF (World Transcription Format) Extension

    The WTF extension provides standardized representation of speech-to-text transcription data from multiple providers.

    Key Features

    • Multi-Provider Support: Whisper, Deepgram, AssemblyAI, Google, Amazon, Azure, and more

    • Standardized Format: Hierarchical structure with transcripts, segments, words, and speakers

    • Quality Metrics: Audio quality assessment and confidence scoring

    • Export Capabilities: SRT and WebVTT subtitle formats

    Adding WTF Transcription Attachments

    Finding WTF Attachments

    Exporting Transcriptions

    Provider Data Conversion

    Advanced WTF Features

    Analysis Tools

    7.3. Extension Validation and Processing

    8. Civic Address Support (vCon 0.3.0)

    Civic addresses provide location information for parties using the GEOPRIV standard.

    9. Party History Events (vCon 0.3.0)

    Track when parties join, leave, or change state during conversations.

    10. Advanced Dialog Features (vCon 0.3.0)

    New dialog fields for enhanced functionality.

    Signing and Verification

    Validation

    Common Patterns and Best Practices

    1. Creating a Complete Conversation

    2. Working with Audio Content

    3. External vs Inline Content

    4. Video Content Handling

    5. Image Content Handling

    6. Content Hashing and Integrity

    Error Handling

    Working with Property Handling Modes

    The Vcon constructor accepts a property_handling parameter to control how non-standard properties are handled:

    LLM-Specific Patterns and Best Practices

    1. Code Generation Templates

    When generating vCon code, use these templates as starting points:

    Basic Conversation Template

    Multimedia Conversation Template

    Extension-Enabled Conversation Template

    Privacy-Compliant Conversation Template

    Transcription-Enabled Conversation Template

    2. Common LLM Tasks

    Converting Chat History to vCon

    Adding AI Analysis to vCon

    Extracting Conversation Data

    Adding Privacy Compliance to vCon

    Converting Provider Transcription to WTF

    Checking Privacy Permissions

    Exporting Transcriptions

    Validating Extensions

    Processing Extensions

    3. Error Handling Patterns

    4. Validation Patterns

    5. Integration Patterns

    REST API Integration

    Database Integration

    6. Performance Considerations

    Conclusion

    The vCon library provides a comprehensive framework for working with conversation data. When generating code:

    1. Start Simple: Begin with Vcon.build_new() and basic Party/Dialog objects

    2. Add Rich Metadata: Use tags, attachments, and analysis for comprehensive data

    3. Handle Multimedia: Leverage video/image processing capabilities when needed

    4. Ensure Security

    The vCon 0.3.0 specification provides a robust foundation for modern conversation data management with support for multimedia content, security, and extensibility.

    Analysis: Results from processing the conversation (sentiment analysis, transcription, etc.)

  • Extensions: Optional features that extend the base vCon functionality

    • Lawful Basis Extension: GDPR-compliant consent management and privacy compliance

    • WTF Extension: World Transcription Format for standardized speech-to-text data

  • Digital Signatures: Cryptographic verification of vCon integrity

  • Civic Addresses: Location information for parties using GEOPRIV standard

  • Party History: Event tracking for multi-party conversations

  • mailto: Email address (e.g., "[email protected]")
    timezone: Party's timezone (e.g., "America/New_York")
    stir: STIR verification for secure telephony
    Custom attributes can be added via kwargs
    "incomplete": Failed or incomplete conversation setup
  • "audio": Audio content

  • "video": Video content

  • audio/x-mp4, audio/ogg, audio/webm
  • audio/x-m4a, audio/aac

  • ,
    video/x-matroska
  • video/mpeg, video/x-flv, video/3gpp, video/x-m4v

  • image/jpeg, image/tiff, application/pdf
  • Content Integrity: Hash validation and canonicalization

  • External Registry Integration: SCITT (Supply Chain Integrity, Transparency, and Trust) support

  • Provider Adapters: Automatic conversion from provider-specific formats

  • Analysis Tools: Keyword extraction, confidence analysis, and transcription comparison

  • : Use digital signatures for integrity verification
  • Validate Always: Check vCon validity before saving or transmitting

  • Handle Errors Gracefully: Implement proper error handling for robust applications

  • Consider Performance: Optimize for storage or processing based on use case

  • Use Extensions: Declare optional features and must-support requirements

  • Track Events: Use party history for complex multi-party conversations

  • Integrate Seamlessly: Follow patterns for API and database integration

  • Implement Privacy Compliance: Use Lawful Basis extension for GDPR compliance

  • Standardize Transcriptions: Use WTF extension for multi-provider transcription support

  • Validate Extensions: Always validate extension data before processing

  • Export Transcriptions: Leverage WTF export capabilities for subtitle formats

  • Check Permissions: Use lawful basis permission checking for privacy compliance

  • # Basic installation
    pip install vcon
    
    # With image processing support (Pillow, PyPDF)
    pip install vcon[image]
    
    # From source
    git clone https://github.com/vcon-dev/vcon-lib.git
    cd vcon-lib
    pip install -e .
    from vcon import Vcon
    
    # Create a new empty vCon
    vcon = Vcon.build_new()
    
    # Create from existing JSON
    vcon = Vcon.build_from_json(json_string)
    
    # Load from file
    vcon = Vcon.load_from_file("conversation.json")
    
    # Load from URL
    vcon = Vcon.load_from_url("https://example.com/conversation.json")
    
    # Generic load (detects if path or URL)
    vcon = Vcon.load("conversation.json")  # or URL
    # Save to file
    vcon.save_to_file("conversation.json")
    
    # Convert to JSON string
    json_str = vcon.to_json()  # or vcon.dumps()
    
    # Convert to dictionary
    vcon_dict = vcon.to_dict()
    
    # Post to URL with optional headers
    response = vcon.post_to_url(
        'https://api.example.com/vcons',
        headers={'x-api-token': 'your-token-here'}
    )
    # Access properties
    uuid = vcon.uuid
    version = vcon.vcon
    created_at = vcon.created_at
    updated_at = vcon.updated_at
    parties_list = vcon.parties
    dialog_list = vcon.dialog
    attachments_list = vcon.attachments
    analysis_list = vcon.analysis
    extensions_list = vcon.extensions
    must_support_list = vcon.must_support
    from vcon.party import Party
    
    # Create a party
    caller = Party(
        tel="+1234567890",
        name="Alice Smith",
        role="caller",
        mailto="[email protected]"
    )
    
    # Add to vCon
    vcon.add_party(caller)
    
    # Find a party by an attribute
    party_index = vcon.find_party_index("name", "Alice Smith")  # Returns index (0-based)
    from vcon.dialog import Dialog
    from datetime import datetime, timezone
    
    # Create a text dialog
    text_dialog = Dialog(
        type="text",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],  # Indices of parties involved
        originator=0,    # Index of the party that sent the message
        mimetype="text/plain",
        body="Hello, I need help with my account."
    )
    
    # Add to vCon
    vcon.add_dialog(text_dialog)
    
    # Create an audio dialog
    audio_dialog = Dialog(
        type="audio",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],
        originator=0,
        mimetype="audio/mp3",
        body=base64_encoded_audio,
        encoding="base64",
        filename="recording.mp3"
    )
    
    vcon.add_dialog(audio_dialog)
    
    # Find a dialog by property
    found_dialog = vcon.find_dialog("type", "text")
    # Add a transfer dialog
    vcon.add_transfer_dialog(
        start=datetime.now(timezone.utc).isoformat(),
        transfer_data={
            "reason": "Call forwarded",
            "from": "+1234567890",
            "to": "+1987654321"
        },
        parties=[0, 1]
    )
    
    # Add an incomplete dialog (for failed conversations)
    vcon.add_incomplete_dialog(
        start=datetime.now(timezone.utc).isoformat(),
        disposition="NO_ANSWER",
        details={"ringDuration": 45000},
        parties=[0, 1]
    )
    # Check dialog type
    is_text = dialog.is_text()
    is_recording = dialog.is_recording()
    is_transfer = dialog.is_transfer()
    is_incomplete = dialog.is_incomplete()
    is_audio = dialog.is_audio()
    is_video = dialog.is_video()
    is_email = dialog.is_email()
    # Add tags
    vcon.add_tag("customer_id", "12345")
    vcon.add_tag("interaction_id", "INT-001")
    
    # Get tag value
    value = vcon.get_tag("customer_id")  # Returns "12345"
    
    # Get all tags
    all_tags = vcon.tags  # Returns the tags attachment dictionary
    # Add an attachment
    vcon.add_attachment(
        type="transcript",
        body="Conversation transcript content...",
        encoding="none"
    )
    
    # Add a base64-encoded attachment
    vcon.add_attachment(
        type="recording",
        body=base64_encoded_content,
        encoding="base64url"
    )
    
    # Find an attachment
    attachment = vcon.find_attachment_by_type("transcript")
    # Add analysis
    vcon.add_analysis(
        type="sentiment",
        dialog=[0],  # Index or indices of dialogs analyzed
        vendor="AnalysisCompany",
        body={"sentiment": "positive", "score": 0.8},
        encoding="json"
    )
    
    # Find analysis
    analysis = vcon.find_analysis_by_type("sentiment")
    # Add extensions used in this vCon
    vcon.add_extension("video")
    vcon.add_extension("encryption")
    vcon.add_extension("sentiment_analysis")
    
    # Add extensions that must be supported by consumers
    vcon.add_must_support("encryption")
    vcon.add_must_support("video")
    
    # Get extensions
    extensions = vcon.get_extensions()  # ['video', 'encryption', 'sentiment_analysis']
    must_support = vcon.get_must_support()  # ['encryption', 'video']
    
    # Remove extensions
    vcon.remove_extension("sentiment_analysis")
    vcon.remove_must_support("video")
    from datetime import datetime, timezone, timedelta
    
    # Add lawful basis attachment
    vcon.add_lawful_basis_attachment(
        lawful_basis="consent",
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=[
            {
                "purpose": "recording",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            },
            {
                "purpose": "analysis",
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat(),
                "conditions": ["anonymized_data_only"]
            }
        ],
        party_index=0,
        dialog_index=0
    )
    
    # Add extension to vCon
    vcon.add_extension("lawful_basis")
    # Check if permission is granted for a specific purpose
    recording_permission = vcon.check_lawful_basis_permission("recording", party_index=0)
    marketing_permission = vcon.check_lawful_basis_permission("marketing", party_index=0)
    
    print(f"Recording permission: {recording_permission}")
    print(f"Marketing permission: {marketing_permission}")
    # Find all lawful basis attachments
    attachments = vcon.find_lawful_basis_attachments()
    
    # Find attachments for a specific party
    party_attachments = vcon.find_lawful_basis_attachments(party_index=0)
    from vcon.extensions.lawful_basis import (
        LawfulBasisAttachment, 
        PurposeGrant, 
        ContentHash,
        ProofMechanism,
        LawfulBasisType,
        ProofType,
        HashAlgorithm
    )
    
    # Create purpose grants with conditions
    purpose_grants = [
        PurposeGrant(
            purpose="recording",
            granted=True,
            granted_at=datetime.now(timezone.utc).isoformat()
        ),
        PurposeGrant(
            purpose="analysis",
            granted=True,
            granted_at=datetime.now(timezone.utc).isoformat(),
            conditions=["anonymized_data_only", "retention_30_days"]
        )
    ]
    
    # Create content hash for integrity
    content_hash = ContentHash(
        algorithm=HashAlgorithm.SHA_256,
        canonicalization="JCS",
        value="computed_hash_value"
    )
    
    # Create proof mechanism
    proof = ProofMechanism(
        proof_type=ProofType.VERBAL_CONFIRMATION,
        timestamp=datetime.now(timezone.utc).isoformat(),
        proof_data={
            "dialog_reference": 0,
            "confirmation_text": "I consent to recording"
        }
    )
    
    # Create comprehensive lawful basis attachment
    attachment = LawfulBasisAttachment(
        lawful_basis=LawfulBasisType.CONSENT,
        expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
        purpose_grants=purpose_grants,
        content_hash=content_hash,
        proof_mechanisms=[proof]
    )
    # Add WTF transcription attachment
    vcon.add_wtf_transcription_attachment(
        transcript={
            "text": "Hello, this is a test transcription.",
            "language": "en",
            "duration": 3.5,
            "confidence": 0.95
        },
        segments=[
            {
                "id": 0,
                "start": 0.0,
                "end": 1.5,
                "text": "Hello, this is",
                "confidence": 0.95,
                "speaker": 0
            },
            {
                "id": 1,
                "start": 1.5,
                "end": 3.5,
                "text": "a test transcription.",
                "confidence": 0.94,
                "speaker": 0
            }
        ],
        metadata={
            "created_at": datetime.now(timezone.utc).isoformat(),
            "processed_at": datetime.now(timezone.utc).isoformat(),
            "provider": "whisper",
            "model": "whisper-1",
            "audio_quality": "high",
            "background_noise": 0.1
        },
        party_index=0,
        dialog_index=0
    )
    
    # Add extension to vCon
    vcon.add_extension("wtf_transcription")
    # Find all WTF attachments
    attachments = vcon.find_wtf_attachments()
    
    # Find attachments for a specific party
    party_attachments = vcon.find_wtf_attachments(party_index=0)
    # Find WTF attachments and export to SRT
    attachments = vcon.find_wtf_attachments(party_index=0)
    if attachments:
        from vcon.extensions.wtf import WTFAttachment
        wtf_attachment = WTFAttachment.from_dict(attachments[0]["body"])
        
        # Export to SRT format
        srt_content = wtf_attachment.export_to_srt()
        print("SRT Export:")
        print(srt_content)
        
        # Export to WebVTT format
        vtt_content = wtf_attachment.export_to_vtt()
        print("WebVTT Export:")
        print(vtt_content)
    from vcon.extensions.wtf import WhisperAdapter, DeepgramAdapter
    
    # Convert Whisper data to WTF format
    whisper_data = {
        "text": "Hello world from Whisper",
        "segments": [
            {
                "start": 0.0,
                "end": 2.0,
                "text": "Hello world from Whisper"
            }
        ]
    }
    
    whisper_adapter = WhisperAdapter()
    wtf_attachment = whisper_adapter.convert(whisper_data)
    
    # Add to vCon
    vcon.add_wtf_transcription_attachment(
        transcript=wtf_attachment.transcript.to_dict(),
        segments=[segment.to_dict() for segment in wtf_attachment.segments],
        metadata=wtf_attachment.metadata.to_dict()
    )
    from vcon.extensions.wtf import (
        WTFAttachment, 
        Transcript, 
        Segment, 
        Word, 
        Speaker,
        Quality,
        Metadata
    )
    
    # Create detailed transcript
    transcript = Transcript(
        text="Hello world",
        language="en",
        duration=2.0,
        confidence=0.95
    )
    
    # Create segments with words
    segments = [
        Segment(
            id=0,
            start=0.0,
            end=2.0,
            text="Hello world",
            confidence=0.95,
            speaker=0,
            words=[
                Word(id=0, start=0.0, end=1.0, text="Hello", confidence=0.95, speaker=0),
                Word(id=1, start=1.0, end=2.0, text="world", confidence=0.95, speaker=0)
            ]
        )
    ]
    
    # Create speaker information
    speakers = [
        Speaker(
            id=0,
            label="Speaker 1",
            segments=[0],
            total_time=2.0,
            confidence=0.9
        )
    ]
    
    # Create quality metrics
    quality = Quality(
        audio_quality="high",
        background_noise=0.1,
        multiple_speakers=False,
        overlapping_speech=False,
        silence_ratio=0.2,
        average_confidence=0.95,
        low_confidence_words=0,
        processing_warnings=[]
    )
    
    # Create metadata
    metadata = Metadata(
        created_at=datetime.now(timezone.utc).isoformat(),
        processed_at=datetime.now(timezone.utc).isoformat(),
        provider="whisper",
        model="whisper-1",
        audio_quality="high",
        background_noise=0.1
    )
    
    # Create comprehensive WTF attachment
    attachment = WTFAttachment(
        transcript=transcript,
        segments=segments,
        metadata=metadata,
        words=[word for segment in segments for word in segment.words],
        speakers=speakers,
        quality=quality
    )
    # Extract keywords from high-confidence words
    keywords = attachment.extract_keywords(min_confidence=0.8)
    
    # Find segments with low confidence
    low_confidence_segments = attachment.find_low_confidence_segments(threshold=0.5)
    
    # Calculate speaking time for each speaker
    speaking_times = attachment.get_speaking_time()
    # Validate all extensions
    validation_results = vcon.validate_extensions()
    print("Extension validation results:")
    for extension, result in validation_results.items():
        if extension != "attachments":
            status = "✓ Valid" if result["is_valid"] else "✗ Invalid"
            print(f"  {extension}: {status}")
            if result["errors"]:
                for error in result["errors"]:
                    print(f"    Error: {error}")
            if result["warnings"]:
                for warning in result["warnings"]:
                    print(f"    Warning: {warning}")
    
    # Process all extensions
    processing_results = vcon.process_extensions()
    print("Extension processing completed")
    from vcon.civic_address import CivicAddress
    
    # Create civic address
    address = CivicAddress(
        country="US",
        a1="CA",  # State
        a3="San Francisco",  # City
        sts="Market Street",  # Street
        hno="123",  # House number
        pc="94102"  # Postal code
    )
    
    # Add to party
    party = Party(
        name="Jane Doe",
        tel="+1555123456",
        civicaddress=address
    )
    
    # Convert to dictionary
    address_dict = address.to_dict()
    from vcon.party import PartyHistory
    from datetime import datetime
    
    # Create party history events
    history = [
        PartyHistory(0, "join", datetime.now()),    # Party 0 joins
        PartyHistory(1, "join", datetime.now()),    # Party 1 joins
        PartyHistory(0, "hold", datetime.now()),    # Party 0 on hold
        PartyHistory(0, "unhold", datetime.now()),  # Party 0 off hold
        PartyHistory(1, "drop", datetime.now())     # Party 1 drops
    ]
    
    # Add to dialog
    dialog = Dialog(
        type="recording",
        start=datetime.now(),
        parties=[0, 1],
        party_history=history
    )
    
    # Valid event types: "join", "drop", "hold", "unhold", "mute", "unmute"
    # Dialog with session tracking and content hashing
    dialog = Dialog(
        type="text",
        start=datetime.now(),
        parties=[0, 1],
        originator=0,
        body="Hello, this is a test message!",
        session_id="session-12345",
        content_hash="c8d3d67f662a787e96e74ccb0a77803138c0f13495a186ccbde495c57c385608",
        application="chat-app",
        message_id="<[email protected]>"
    )
    
    # Video dialog with metadata
    video_dialog = Dialog(
        type="video",
        start=datetime.now(),
        parties=[0, 1],
        mimetype="video/mp4",
        resolution="1920x1080",
        frame_rate=30.0,
        codec="H.264",
        bitrate=5000000,
        filename="recording.mp4"
    )
    
    # Incomplete dialog with disposition
    incomplete_dialog = Dialog(
        type="incomplete",
        start=datetime.now(),
        parties=[0],
        disposition="no-answer"  # Valid: no-answer, congestion, failed, busy, hung-up, voicemail-no-message
    )
    # Generate a key pair
    private_key, public_key = Vcon.generate_key_pair()
    
    # Sign the vCon
    vcon.sign(private_key)
    
    # Verify the signature
    is_valid = vcon.verify(public_key)
    # Validate a vCon object
    is_valid, errors = vcon.is_valid()
    if not is_valid:
        print("Validation errors:", errors)
    
    # Validate a file
    is_valid, errors = Vcon.validate_file("conversation.json")
    
    # Validate a JSON string
    is_valid, errors = Vcon.validate_json(json_string)
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime, timezone
    
    # Create a new vCon
    vcon = Vcon.build_new()
    
    # Add participants
    caller = Party(tel="+1234567890", name="Alice", role="caller")
    agent = Party(tel="+1987654321", name="Bob", role="agent")
    vcon.add_party(caller)
    vcon.add_party(agent)
    
    # Add conversation dialogs in sequence
    vcon.add_dialog(Dialog(
        type="text",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],
        originator=0,  # Caller
        mimetype="text/plain",
        body="Hello, I need help with my account."
    ))
    
    vcon.add_dialog(Dialog(
        type="text",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],
        originator=1,  # Agent
        mimetype="text/plain",
        body="I'd be happy to help. Can you provide your account number?"
    ))
    
    # Add metadata
    vcon.add_tag("customer_id", "12345")
    vcon.add_tag("interaction_id", "INT-001")
    
    # Validate and save
    is_valid, errors = vcon.is_valid()
    if is_valid:
        vcon.save_to_file("conversation.json")
    else:
        print("Validation errors:", errors)
    import base64
    
    # Reading an audio file and adding it to a dialog
    with open("recording.mp3", "rb") as f:
        audio_data = f.read()
        audio_base64 = base64.b64encode(audio_data).decode("utf-8")
    
    audio_dialog = Dialog(
        type="audio",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],
        originator=0,
        mimetype="audio/mp3",
        body=audio_base64,
        encoding="base64",
        filename="recording.mp3"
    )
    vcon.add_dialog(audio_dialog)
    # External content (referenced by URL)
    external_dialog = Dialog(
        type="recording",
        start=datetime.now(timezone.utc).isoformat(),
        parties=[0, 1],
        url="https://example.com/recordings/call123.mp3",
        mimetype="audio/mp3"
    )
    
    # Check if dialog refers to external content
    if external_dialog.is_external_data():
        # Convert to inline data
        external_dialog.to_inline_data()
    
    # Check if dialog contains inline data
    if dialog.is_inline_data():
        print("Dialog contains embedded content")
    # Add video data with metadata
    video_dialog = Dialog(
        type="video",
        start=datetime.now(),
        parties=[0, 1],
        mimetype="video/mp4",
        resolution="1920x1080",
        frame_rate=30.0,
        codec="H.264"
    )
    
    # Add video data (inline or external)
    video_dialog.add_video_data(
        video_data=binary_video_data,  # or URL string
        filename="recording.mp4",
        mimetype="video/mp4",
        inline=True,  # False for external reference
        metadata={"duration": 120, "quality": "high"}
    )
    
    # Extract video metadata using FFmpeg
    metadata = video_dialog.extract_video_metadata()
    
    # Generate thumbnail
    thumbnail_data = video_dialog.generate_thumbnail(
        timestamp=10.0,  # Time in seconds
        width=320,
        height=240,
        quality=90
    )
    
    # Transcode video to different format
    video_dialog.transcode_video(
        target_format="webm",
        codec="vp9",
        bit_rate=2000000,
        width=1280,
        height=720
    )
    # Add image data from file
    image_dialog = Dialog(
        type="text",  # Can be any type
        start=datetime.now(),
        parties=[0, 1]
    )
    
    # Add image from file
    image_dialog.add_image_data(
        image_path="screenshot.png",
        mimetype="image/jpeg"  # Optional, auto-detected if not provided
    )
    
    # Generate thumbnail
    thumbnail_b64 = image_dialog.generate_thumbnail(max_size=(200, 200))
    
    # Check if dialog has image content
    if image_dialog.is_image():
        print("Dialog contains image content")
    
    # Check for PDF content
    if image_dialog.is_pdf():
        print("Dialog contains PDF content")
    # Calculate content hash
    content_hash = dialog.calculate_content_hash("sha256")
    
    # Set content hash for external files
    dialog.set_content_hash(content_hash)
    
    # Verify content integrity
    is_valid = dialog.verify_content_hash(expected_hash, "sha256")
    
    # Check if external data has changed
    if dialog.is_external_data_changed():
        print("External content has been modified")
    try:
        vcon = Vcon.load_from_file("conversation.json")
        is_valid, errors = vcon.is_valid()
        if not is_valid:
            print("Validation errors:", errors)
    except FileNotFoundError:
        print("File not found")
    except json.JSONDecodeError:
        print("Invalid JSON format")
    except Exception as e:
        print(f"Error: {str(e)}")
    # Default mode: keep non-standard properties
    vcon = Vcon(vcon_dict)  # or Vcon(vcon_dict, property_handling="default")
    
    # Strict mode: remove non-standard properties
    vcon = Vcon(vcon_dict, property_handling="strict")
    
    # Meta mode: move non-standard properties to meta object
    vcon = Vcon(vcon_dict, property_handling="meta")
    from vcon import Vcon
    from vcon.party import Party
    from vcon.dialog import Dialog
    from datetime import datetime
    
    def create_basic_conversation():
        # Create vCon
        vcon = Vcon.build_new()
        
        # Add parties
        caller = Party(tel="+1234567890", name="Caller", role="caller")
        agent = Party(tel="+1987654321", name="Agent", role="agent")
        vcon.add_party(caller)
        vcon.add_party(agent)
        
        # Add conversation
        vcon.add_dialog(Dialog(
            type="text",
            start=datetime.now().isoformat(),
            parties=[0, 1],
            originator=0,
            body="Hello, I need help."
        ))
        
        return vcon
    def create_multimedia_conversation():
        vcon = Vcon.build_new()
        
        # Add parties with enhanced contact info
        caller = Party(
            tel="+1234567890",
            name="Alice",
            role="caller",
            mailto="[email protected]",
            timezone="America/New_York"
        )
        agent = Party(
            tel="+1987654321", 
            name="Bob",
            role="agent",
            sip="sip:[email protected]"
        )
        vcon.add_party(caller)
        vcon.add_party(agent)
        
        # Add text dialog
        vcon.add_dialog(Dialog(
            type="text",
            start=datetime.now().isoformat(),
            parties=[0, 1],
            originator=0,
            body="Hello, I need help with my account."
        ))
        
        # Add audio dialog
        vcon.add_dialog(Dialog(
            type="recording",
            start=datetime.now().isoformat(),
            parties=[0, 1],
            mimetype="audio/mp3",
            filename="conversation.mp3"
        ))
        
        # Add analysis
        vcon.add_analysis(
            type="sentiment",
            dialog=[0, 1],
            vendor="SentimentAnalyzer",
            body={"sentiment": "positive", "confidence": 0.85},
            encoding="json"
        )
        
        return vcon
    def create_extension_enabled_conversation():
        """Create a vCon with both Lawful Basis and WTF extensions."""
        from datetime import datetime, timezone, timedelta
        
        vcon = Vcon.build_new()
        
        # Add parties
        caller = Party(
            tel="+1234567890",
            name="Alice",
            role="caller",
            mailto="[email protected]"
        )
        agent = Party(
            tel="+1987654321",
            name="Bob", 
            role="agent",
            sip="sip:[email protected]"
        )
        vcon.add_party(caller)
        vcon.add_party(agent)
        
        # Add recording dialog
        dialog = Dialog(
            type="recording",
            start=datetime.now(timezone.utc),
            parties=[0, 1],
            mimetype="audio/mp3"
        )
        vcon.add_dialog(dialog)
        
        # Add lawful basis for consent
        vcon.add_lawful_basis_attachment(
            lawful_basis="consent",
            expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
            purpose_grants=[
                {
                    "purpose": "recording",
                    "granted": True,
                    "granted_at": datetime.now(timezone.utc).isoformat()
                },
                {
                    "purpose": "transcription",
                    "granted": True,
                    "granted_at": datetime.now(timezone.utc).isoformat()
                }
            ],
            party_index=0
        )
        
        # Add transcription
        vcon.add_wtf_transcription_attachment(
            transcript={
                "text": "Hello, I need help with my account.",
                "language": "en",
                "duration": 4.2,
                "confidence": 0.92
            },
            segments=[
                {
                    "id": 0,
                    "start": 0.0,
                    "end": 4.2,
                    "text": "Hello, I need help with my account.",
                    "confidence": 0.92,
                    "speaker": 0
                }
            ],
            metadata={
                "created_at": datetime.now(timezone.utc).isoformat(),
                "processed_at": datetime.now(timezone.utc).isoformat(),
                "provider": "whisper",
                "model": "whisper-1"
            },
            party_index=0,
            dialog_index=0
        )
        
        # Add extensions
        vcon.add_extension("lawful_basis")
        vcon.add_extension("wtf_transcription")
        
        return vcon
    def create_privacy_compliant_conversation():
        """Create a vCon with comprehensive privacy compliance."""
        from datetime import datetime, timezone, timedelta
        from vcon.extensions.lawful_basis import (
            LawfulBasisAttachment, 
            PurposeGrant, 
            ContentHash,
            ProofMechanism,
            LawfulBasisType,
            ProofType,
            HashAlgorithm
        )
        
        vcon = Vcon.build_new()
        
        # Add parties
        caller = Party(
            tel="+1234567890",
            name="Alice",
            role="caller"
        )
        agent = Party(
            tel="+1987654321",
            name="Bob",
            role="agent"
        )
        vcon.add_party(caller)
        vcon.add_party(agent)
        
        # Add dialog
        dialog = Dialog(
            type="recording",
            start=datetime.now(timezone.utc),
            parties=[0, 1],
            mimetype="audio/mp3"
        )
        vcon.add_dialog(dialog)
        
        # Create comprehensive lawful basis
        purpose_grants = [
            PurposeGrant(
                purpose="recording",
                granted=True,
                granted_at=datetime.now(timezone.utc).isoformat()
            ),
            PurposeGrant(
                purpose="analysis",
                granted=True,
                granted_at=datetime.now(timezone.utc).isoformat(),
                conditions=["anonymized_data_only", "retention_30_days"]
            ),
            PurposeGrant(
                purpose="marketing",
                granted=False,
                granted_at=datetime.now(timezone.utc).isoformat()
            )
        ]
        
        # Create content hash for integrity
        content_hash = ContentHash(
            algorithm=HashAlgorithm.SHA_256,
            canonicalization="JCS",
            value="computed_hash_value"
        )
        
        # Create proof mechanism
        proof = ProofMechanism(
            proof_type=ProofType.VERBAL_CONFIRMATION,
            timestamp=datetime.now(timezone.utc).isoformat(),
            proof_data={
                "dialog_reference": 0,
                "confirmation_text": "I consent to recording for quality assurance"
            }
        )
        
        # Create lawful basis attachment
        attachment = LawfulBasisAttachment(
            lawful_basis=LawfulBasisType.CONSENT,
            expiration=(datetime.now(timezone.utc) + timedelta(days=365)).isoformat(),
            purpose_grants=purpose_grants,
            content_hash=content_hash,
            proof_mechanisms=[proof]
        )
        
        # Add to vCon
        vcon.vcon_dict["attachments"].append({
            "type": "lawful_basis",
            "encoding": "json",
            "body": attachment.to_dict(),
            "party": 0,
            "dialog": 0
        })
        
        # Add extension
        vcon.add_extension("lawful_basis")
        
        return vcon
    def create_transcription_enabled_conversation():
        """Create a vCon with comprehensive transcription support."""
        from datetime import datetime, timezone
        from vcon.extensions.wtf import (
            WTFAttachment, 
            Transcript, 
            Segment, 
            Word, 
            Speaker,
            Quality,
            Metadata
        )
        
        vcon = Vcon.build_new()
        
        # Add parties
        caller = Party(
            tel="+1234567890",
            name="Alice",
            role="caller"
        )
        agent = Party(
            tel="+1987654321",
            name="Bob",
            role="agent"
        )
        vcon.add_party(caller)
        vcon.add_party(agent)
        
        # Add dialog
        dialog = Dialog(
            type="recording",
            start=datetime.now(timezone.utc),
            parties=[0, 1],
            mimetype="audio/mp3"
        )
        vcon.add_dialog(dialog)
        
        # Create detailed transcript
        transcript = Transcript(
            text="Hello, I need help with my account. Can you assist me?",
            language="en",
            duration=6.5,
            confidence=0.94
        )
        
        # Create segments with words
        segments = [
            Segment(
                id=0,
                start=0.0,
                end=3.2,
                text="Hello, I need help with my account.",
                confidence=0.95,
                speaker=0,
                words=[
                    Word(id=0, start=0.0, end=0.5, text="Hello", confidence=0.98, speaker=0),
                    Word(id=1, start=0.5, end=0.8, text="I", confidence=0.95, speaker=0),
                    Word(id=2, start=0.8, end=1.1, text="need", confidence=0.92, speaker=0),
                    Word(id=3, start=1.1, end=1.4, text="help", confidence=0.94, speaker=0),
                    Word(id=4, start=1.4, end=1.7, text="with", confidence=0.90, speaker=0),
                    Word(id=5, start=1.7, end=2.0, text="my", confidence=0.96, speaker=0),
                    Word(id=6, start=2.0, end=2.5, text="account", confidence=0.93, speaker=0)
                ]
            ),
            Segment(
                id=1,
                start=3.2,
                end=6.5,
                text="Can you assist me?",
                confidence=0.92,
                speaker=0,
                words=[
                    Word(id=7, start=3.2, end=3.5, text="Can", confidence=0.91, speaker=0),
                    Word(id=8, start=3.5, end=3.8, text="you", confidence=0.94, speaker=0),
                    Word(id=9, start=3.8, end=4.2, text="assist", confidence=0.89, speaker=0),
                    Word(id=10, start=4.2, end=4.5, text="me", confidence=0.95, speaker=0)
                ]
            )
        ]
        
        # Create speaker information
        speakers = [
            Speaker(
                id=0,
                label="Customer",
                segments=[0, 1],
                total_time=6.5,
                confidence=0.93
            )
        ]
        
        # Create quality metrics
        quality = Quality(
            audio_quality="high",
            background_noise=0.05,
            multiple_speakers=False,
            overlapping_speech=False,
            silence_ratio=0.1,
            average_confidence=0.94,
            low_confidence_words=2,
            processing_warnings=[]
        )
        
        # Create metadata
        metadata = Metadata(
            created_at=datetime.now(timezone.utc).isoformat(),
            processed_at=datetime.now(timezone.utc).isoformat(),
            provider="whisper",
            model="whisper-1",
            audio_quality="high",
            background_noise=0.05
        )
        
        # Create comprehensive WTF attachment
        attachment = WTFAttachment(
            transcript=transcript,
            segments=segments,
            metadata=metadata,
            words=[word for segment in segments for word in segment.words],
            speakers=speakers,
            quality=quality
        )
        
        # Add to vCon
        vcon.vcon_dict["attachments"].append({
            "type": "wtf_transcription",
            "encoding": "json",
            "body": attachment.to_dict(),
            "party": 0,
            "dialog": 0
        })
        
        # Add extension
        vcon.add_extension("wtf_transcription")
        
        return vcon
    def chat_to_vcon(chat_messages, participants):
        vcon = Vcon.build_new()
        
        # Add participants as parties
        party_map = {}
        for i, participant in enumerate(participants):
            party = Party(
                name=participant.get("name", f"User {i}"),
                role=participant.get("role", "participant")
            )
            vcon.add_party(party)
            party_map[participant["id"]] = i
        
        # Add messages as dialogs
        for message in chat_messages:
            vcon.add_dialog(Dialog(
                type="text",
                start=message["timestamp"],
                parties=[party_map[message["sender_id"]]],
                originator=party_map[message["sender_id"]],
                body=message["content"]
            ))
        
        return vcon
    def add_ai_analysis(vcon, analysis_type, results, dialog_indices=None):
        if dialog_indices is None:
            dialog_indices = list(range(len(vcon.dialog)))
        
        vcon.add_analysis(
            type=analysis_type,
            dialog=dialog_indices,
            vendor="AI-Analyzer",
            body=results,
            encoding="json"
        )
        
        # Add extension if using AI features
        vcon.add_extension("ai_analysis")
    def extract_conversation_data(vcon):
        data = {
            "uuid": vcon.uuid,
            "created_at": vcon.created_at,
            "parties": [],
            "dialogs": [],
            "analysis": []
        }
        
        # Extract parties
        for party in vcon.parties:
            data["parties"].append({
                "name": getattr(party, "name", None),
                "role": getattr(party, "role", None),
                "tel": getattr(party, "tel", None)
            })
        
        # Extract dialogs
        for dialog in vcon.dialog:
            data["dialogs"].append({
                "type": dialog.get("type"),
                "start": dialog.get("start"),
                "body": dialog.get("body", "")[:100] + "..." if len(dialog.get("body", "")) > 100 else dialog.get("body", ""),
                "parties": dialog.get("parties", [])
            })
        
        # Extract analysis
        for analysis in vcon.analysis:
            data["analysis"].append({
                "type": analysis.get("type"),
                "vendor": analysis.get("vendor"),
                "dialog_count": len(analysis.get("dialog", []))
            })
        
        return data
    def add_privacy_compliance(vcon, party_index, purposes, expiration_days=365):
        """Add lawful basis attachment for privacy compliance."""
        from datetime import datetime, timezone, timedelta
        
        purpose_grants = []
        for purpose in purposes:
            purpose_grants.append({
                "purpose": purpose,
                "granted": True,
                "granted_at": datetime.now(timezone.utc).isoformat()
            })
        
        vcon.add_lawful_basis_attachment(
            lawful_basis="consent",
            expiration=(datetime.now(timezone.utc) + timedelta(days=expiration_days)).isoformat(),
            purpose_grants=purpose_grants,
            party_index=party_index
        )
        
        vcon.add_extension("lawful_basis")
        return vcon
    def convert_provider_transcription(vcon, provider_data, provider_type, party_index=0, dialog_index=0):
        """Convert provider-specific transcription data to WTF format."""
        from vcon.extensions.wtf import WhisperAdapter, DeepgramAdapter, AssemblyAIAdapter
        
        # Select appropriate adapter
        adapters = {
            "whisper": WhisperAdapter(),
            "deepgram": DeepgramAdapter(),
            "assemblyai": AssemblyAIAdapter()
        }
        
        if provider_type not in adapters:
            raise ValueError(f"Unsupported provider: {provider_type}")
        
        # Convert to WTF format
        adapter = adapters[provider_type]
        wtf_attachment = adapter.convert(provider_data)
        
        # Add to vCon
        vcon.add_wtf_transcription_attachment(
            transcript=wtf_attachment.transcript.to_dict(),
            segments=[segment.to_dict() for segment in wtf_attachment.segments],
            metadata=wtf_attachment.metadata.to_dict(),
            party_index=party_index,
            dialog_index=dialog_index
        )
        
        vcon.add_extension("wtf_transcription")
        return vcon
    def check_privacy_permissions(vcon, party_index, purposes):
        """Check if party has permission for specific purposes."""
        results = {}
        for purpose in purposes:
            results[purpose] = vcon.check_lawful_basis_permission(purpose, party_index)
        return results
    def export_transcriptions(vcon, party_index=None, format="srt"):
        """Export transcriptions from vCon to various formats."""
        attachments = vcon.find_wtf_attachments(party_index)
        exports = []
        
        for attachment in attachments:
            from vcon.extensions.wtf import WTFAttachment
            wtf_attachment = WTFAttachment.from_dict(attachment["body"])
            
            if format.lower() == "srt":
                content = wtf_attachment.export_to_srt()
            elif format.lower() == "vtt":
                content = wtf_attachment.export_to_vtt()
            else:
                raise ValueError(f"Unsupported format: {format}")
            
            exports.append({
                "party_index": attachment.get("party"),
                "dialog_index": attachment.get("dialog"),
                "format": format,
                "content": content
            })
        
        return exports
    def validate_vcon_extensions(vcon):
        """Validate all extensions in a vCon and return detailed results."""
        validation_results = vcon.validate_extensions()
        
        summary = {
            "valid": True,
            "extensions": {},
            "errors": [],
            "warnings": []
        }
        
        for extension, result in validation_results.items():
            if extension != "attachments":
                summary["extensions"][extension] = {
                    "valid": result["is_valid"],
                    "errors": result["errors"],
                    "warnings": result["warnings"]
                }
                
                if not result["is_valid"]:
                    summary["valid"] = False
                    summary["errors"].extend(result["errors"])
                
                summary["warnings"].extend(result["warnings"])
        
        return summary
    def process_vcon_extensions(vcon):
        """Process all extensions in a vCon and return results."""
        processing_results = vcon.process_extensions()
        
        summary = {
            "success": True,
            "results": processing_results,
            "errors": []
        }
        
        # Check for processing errors
        if "error" in processing_results:
            summary["success"] = False
            summary["errors"].append(processing_results["error"])
        
        return summary
    def safe_vcon_operation(operation_func, *args, **kwargs):
        """Safely execute vCon operations with proper error handling."""
        try:
            return operation_func(*args, **kwargs)
        except ValueError as e:
            return {"error": f"Validation error: {str(e)}", "success": False}
        except FileNotFoundError as e:
            return {"error": f"File not found: {str(e)}", "success": False}
        except Exception as e:
            return {"error": f"Unexpected error: {str(e)}", "success": False}
    
    # Usage
    result = safe_vcon_operation(Vcon.load, "conversation.json")
    if not result.get("success", True):
        print(f"Error: {result['error']}")
    def validate_and_fix_vcon(vcon):
        """Validate vCon and attempt to fix common issues."""
        is_valid, errors = vcon.is_valid()
        
        if is_valid:
            return {"valid": True, "errors": []}
        
        fixes_applied = []
        
        # Fix common issues
        for error in errors:
            if "missing uuid" in error.lower():
                # UUID is auto-generated, this shouldn't happen
                pass
            elif "invalid dialog type" in error.lower():
                # Try to fix invalid dialog types
                for dialog in vcon.dialog:
                    if dialog.get("type") not in ["text", "recording", "transfer", "incomplete", "audio", "video"]:
                        dialog["type"] = "text"  # Default to text
                        fixes_applied.append(f"Fixed invalid dialog type: {dialog.get('type')}")
        
        # Re-validate after fixes
        is_valid, remaining_errors = vcon.is_valid()
        
        return {
            "valid": is_valid,
            "errors": remaining_errors,
            "fixes_applied": fixes_applied
        }
    def vcon_to_api_payload(vcon):
        """Convert vCon to API payload format."""
        return {
            "vcon": vcon.to_dict(),
            "metadata": {
                "version": vcon.vcon,
                "created_at": vcon.created_at,
                "party_count": len(vcon.parties),
                "dialog_count": len(vcon.dialog)
            }
        }
    
    def api_payload_to_vcon(payload):
        """Convert API payload to vCon."""
        return Vcon(payload["vcon"])
    def vcon_to_database_record(vcon):
        """Convert vCon to database record format."""
        return {
            "id": vcon.uuid,
            "version": vcon.vcon,
            "created_at": vcon.created_at,
            "updated_at": vcon.updated_at,
            "data": vcon.to_json(),
            "party_count": len(vcon.parties),
            "dialog_count": len(vcon.dialog),
            "has_attachments": len(vcon.attachments) > 0,
            "has_analysis": len(vcon.analysis) > 0
        }
    def optimize_vcon_for_storage(vcon):
        """Optimize vCon for storage by converting large content to external references."""
        for i, dialog in enumerate(vcon.dialog):
            if dialog.get("body") and len(dialog["body"]) > 1000000:  # 1MB threshold
                # In real implementation, upload to storage and get URL
                external_url = f"https://storage.example.com/dialog_{i}_content"
                dialog["url"] = external_url
                dialog["content_hash"] = dialog.calculate_content_hash()
                del dialog["body"]
                dialog["encoding"] = None
    
    def optimize_vcon_for_processing(vcon):
        """Optimize vCon for processing by loading external content."""
        for dialog in vcon.dialog:
            if dialog.is_external_data():
                try:
                    dialog.to_inline_data()
                except Exception as e:
                    print(f"Failed to load external content: {e}")