src.core.persistence module#

General persistence layer for Marcus systems.

Provides a unified storage interface for Events, Context, Memory, and other systems that need persistent data. Supports multiple backends starting with file-based storage and ready for database backends.

class src.core.persistence.PersistenceBackend[source]#

Bases: object

Base class for persistence backends.

async store(collection, key, data)[source]#

Store data in a collection.

Return type:

None

Parameters:
async retrieve(collection, key)[source]#

Retrieve data from a collection.

Return type:

Optional[Dict[str, Any]]

Parameters:
async query(collection, filter_func=None, limit=100)[source]#

Query data from a collection with optional filtering.

Return type:

List[Dict[str, Any]]

Parameters:

  • collection (str)

  • filter_func (Any | None)

  • limit (int)

async delete(collection, key)[source]#

Delete data from a collection.

Return type:

None

Parameters:
async clear_old(collection, days)[source]#

Clear data older than specified days.

Return type:

int

Parameters:
class src.core.persistence.FilePersistence[source]#

Bases: PersistenceBackend

File-based persistence using JSON files.

__init__(storage_dir=None)[source]#

Initialize file persistence.

Parameters:

storage_dir (Path | None)

Return type:

None

async store(collection, key, data)[source]#

Store data in a collection.

Return type:

None

Parameters:
async retrieve(collection, key)[source]#

Retrieve data from a collection.

Return type:

Optional[Dict[str, Any]]

Parameters:
async query(collection, filter_func=None, limit=100)[source]#

Query data from a collection.

Return type:

List[Dict[str, Any]]

Parameters:

  • collection (str)

  • filter_func (Any | None)

  • limit (int)

async delete(collection, key)[source]#

Delete data from a collection.

Return type:

None

Parameters:
async clear_old(collection, days)[source]#

Clear data older than specified days.

Return type:

int

Parameters:
async calculate_median_task_duration()[source]#

Calculate median task duration (fallback for file-based persistence).

Loads task outcomes from file and calculates median in memory. Less efficient than SQL-based approach but works with file backend.

Returns:

Median task duration in hours. Returns 1.0 if no data available.

Return type:

float

class src.core.persistence.SQLitePersistence[source]#

Bases: PersistenceBackend

SQLite-based persistence for better performance and queries.

__init__(db_path=None)[source]#

Initialize SQLite persistence.

Parameters:

db_path (Path | None)

Return type:

None

async store(collection, key, data)[source]#

Store data in SQLite.

Return type:

None

Parameters:
async retrieve(collection, key)[source]#

Retrieve data from SQLite.

Return type:

Optional[Dict[str, Any]]

Parameters:
async query(collection, filter_func=None, limit=100)[source]#

Query data from SQLite.

Return type:

List[Dict[str, Any]]

Parameters:

  • collection (str)

  • filter_func (Any | None)

  • limit (int)

async delete(collection, key)[source]#

Delete data from SQLite.

Return type:

None

Parameters:
async clear_old(collection, days)[source]#

Clear old data from SQLite.

Return type:

int

Parameters:
async calculate_median_task_duration()[source]#

Calculate median task duration from all successful completed tasks.

Uses SQL to efficiently compute median across all historical data. This is more scalable than loading all outcomes into memory.

Returns:

Median task duration in hours. Returns 1.0 if no data available.

Return type:

float

class src.core.persistence.Persistence[source]#

Bases: object

Main persistence interface for Marcus systems.

Provides a unified way to store and retrieve data for Events, Context, Memory, and other systems.

__init__(backend=None)[source]#

Initialize persistence layer.

Parameters:

backend (Optional[PersistenceBackend]) – Storage backend to use. Defaults to FilePersistence.

Return type:

None

async store_event(event)[source]#

Store an event.

Return type:

None

Parameters:

event (Event)

async get_events(event_type=None, source=None, limit=100)[source]#

Retrieve events with optional filtering.

Return type:

List[Event]

Parameters:

  • event_type (str | None)

  • source (str | None)

  • limit (int)

async store_decision(decision)[source]#

Store an architectural decision.

Return type:

None

Parameters:

decision (Decision)

async get_decisions(task_id=None, agent_id=None, limit=50)[source]#

Retrieve decisions with optional filtering.

Return type:

List[Decision]

Parameters:

  • task_id (str | None)

  • agent_id (str | None)

  • limit (int)

async store(collection, key, data)[source]#

Store arbitrary data in a collection.

Return type:

None

Parameters:
async retrieve(collection, key)[source]#

Retrieve arbitrary data from a collection.

Return type:

Optional[Dict[str, Any]]

Parameters:
async query(collection, filter_func=None, limit=100)[source]#

Query a collection.

Return type:

List[Dict[str, Any]]

Parameters:

  • collection (str)

  • filter_func (Any | None)

  • limit (int)

async delete(collection, key)[source]#

Delete data from a collection.

Return type:

None

Parameters:
async cleanup(days=30)[source]#

Clean up old data from all collections.

Return type:

Dict[str, int]

Parameters:

days (int)

class src.core.persistence.MemoryPersistence[source]#

Bases: PersistenceBackend

In-memory persistence for testing and temporary storage.

__init__()[source]#

Initialize memory persistence.

Return type:

None

async store(collection, key, data)[source]#

Store data in memory.

Return type:

None

Parameters:
async retrieve(collection, key)[source]#

Retrieve data from memory.

Return type:

Optional[Dict[str, Any]]

Parameters:
async query(collection, filter_func=None, limit=100)[source]#

Query data from memory.

Return type:

List[Dict[str, Any]]

Parameters:

  • collection (str)

  • filter_func (Any | None)

  • limit (int)

async delete(collection, key)[source]#

Delete data from memory.

Return type:

None

Parameters:
async clear_old(collection, days)[source]#

Clear old data from memory.

Return type:

int

Parameters: