Indexing Module API¶
The indexing module provides powerful multi-dimensional indexing capabilities for creating variables and constraints indexed by one or more data models with type safety and filtering support.
Overview¶
The indexing module enables data-driven modeling through two key components that work together to create flexible, type-safe multi-dimensional index spaces:
graph LR
A[Data Models] --> B[LXIndexDimension]
B --> C[LXCartesianProduct]
C --> D[Multi-Model Variables]
C --> E[Multi-Model Constraints]
F[Filters] --> B
G[Cross-Filters] --> C
style A fill:#e1f5ff
style B fill:#fff4e1
style C fill:#ffe1e1
style D fill:#e1ffe1
style E fill:#f0e1ff
Key Features:
Type-Safe: Full generic type support with IDE autocomplete
Data-Driven: Automatic expansion based on data instances
Filtering: Both per-dimension and cross-dimension filtering
Sparse Indexing: Only create variables/constraints for valid combinations
ORM Integration: Works with direct data or ORM queries
Components¶
Index Dimension¶
Represents a single dimension in multi-dimensional model indexing. |
The LXIndexDimension class represents a single dimension
in multi-dimensional indexing with:
Model type specification
Index key extraction
Per-dimension filtering
Data source configuration (direct data or ORM)
Cartesian Product¶
Type-safe cartesian product of multiple index dimensions. |
The LXCartesianProduct class combines multiple dimensions
to create multi-dimensional index spaces with:
Combination of 2+ dimensions
Cross-dimension filtering
Tuple-based indexing
Sparse index generation
Detailed API Reference¶
LXIndexDimension¶
Index dimension definitions for multi-dimensional model indexing.
This module provides the LXIndexDimension class, which represents a single dimension in multi-dimensional indexing. Index dimensions can be combined via cartesian products to create multi-model indexed variables and constraints.
- Classes:
- LXIndexDimension: A single dimension of a multi-dimensional index with filtering
and data source configuration
Example
Creating a dimension with filtering:
from lumix import LXIndexDimension
driver_dim = (
LXIndexDimension(Driver, lambda d: d.id)
.from_data(drivers)
.where(lambda d: d.is_active and d.years_experience >= 2)
)
Using with ORM:
driver_dim = (
LXIndexDimension(Driver, lambda d: d.id)
.from_model(session)
.where(lambda d: d.is_active)
)
See also
LXCartesianProduct: Combines dimensionsLXVariable: Uses dimensions for indexing
- class lumix.indexing.dimensions.LXIndexDimension(model_type, key_func, filter_func=None, label=None, _data=None, _session=None)[source]
Bases:
Generic[TModel]Represents a single dimension in multi-dimensional model indexing.
An LXIndexDimension defines one dimension of a potentially multi-dimensional index space. It encapsulates the model type, indexing function, optional filters, and data source for that dimension. Dimensions can be combined via LXCartesianProduct to create multi-model indexed variables and constraints.
This class is central to LumiX’s data-driven modeling approach, enabling automatic expansion of variables and constraints across data instances with type safety and IDE support.
- Type Parameters:
TModel: The data model type for this dimension (e.g., Driver, Product, Date)
- Parameters:
- model_type
The Python class representing the data model for this dimension
- key_func
Function to extract the index key from a model instance
- filter_func
Optional predicate to filter which instances to include
- label
Optional human-readable label for this dimension
- _data
Direct data instances (mutually exclusive with _session)
- _session
ORM session for querying instances (mutually exclusive with _data)
Examples
Basic dimension with direct data:
driver_dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_data(drivers) )
Dimension with filtering:
active_driver_dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_data(drivers) .where(lambda d: d.is_active and d.years_experience >= 2) )
Dimension with ORM:
product_dim = ( LXIndexDimension(Product, lambda p: p.sku) .from_model(db_session) .where(lambda p: p.in_stock) )
Compound key extraction:
route_dim = ( LXIndexDimension(Route, lambda r: (r.origin, r.destination)) .from_data(routes) )
See also
LXCartesianProduct: Combines dimensionsindexed_by_product: Uses dimensionsDriver Scheduling Example (examples/02_driver_scheduling): Real-world usage
Note
Index dimensions follow the “late binding” pattern - data instances are not retrieved until the dimension is actually used during model solving. This allows dimensions to be defined before data is available and supports dynamic data sources.
- __deepcopy__(memo)[source]
Custom deepcopy that detaches ORM sessions and handles lambda closures.
This method enables what-if analysis on models using ORM data sources by: 1. Materializing lazy-loaded ORM data before copying 2. Detaching ORM objects from database sessions 3. Safely copying lambda functions that may capture ORM objects
- Parameters:
memo – Dictionary for tracking circular references during deepcopy
- Returns:
Deep copy of this dimension with all ORM dependencies resolved
Note
After copying, the new dimension will have _session=None and all data stored in _data as detached objects safe for pickling.
- __getstate__()[source]
Support for pickle protocol - detach ORM sessions before pickling.
- Returns:
Dictionary of instance state safe for pickling
- __setstate__(state)[source]
Support for pickle protocol - restore from pickled state.
- Parameters:
state – Dictionary of instance state from pickling
- from_data(data)[source]
Provide data instances directly for this dimension.
This method configures the dimension to use a pre-existing list of model instances. Use this when you have data already loaded in memory or when not using an ORM.
- Parameters:
data (
List[TypeVar(TModel)]) – List of model instances for this dimension- Return type:
LXIndexDimension[TypeVar(TModel)]- Returns:
Self for method chaining
Examples
Basic usage:
drivers = [Driver("D1", "Alice"), Driver("D2", "Bob")] dim = LXIndexDimension(Driver, lambda d: d.id).from_data(drivers)
With filtering:
dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_data(drivers) .where(lambda d: d.is_active) )
Note
This method is mutually exclusive with from_model(). If both are called, the last call takes precedence.
- from_model(session)[source]
Configure dimension to query data from an ORM session.
This method configures the dimension to query instances from a database using an ORM session (e.g., SQLAlchemy, Django ORM). The actual query is executed lazily when get_instances() is called.
- Parameters:
session (
Any) – ORM session object (SQLAlchemy Session, Django ORM manager, etc.)- Return type:
LXIndexDimension[TypeVar(TModel)]- Returns:
Self for method chaining
Examples
SQLAlchemy session:
from sqlalchemy.orm import Session dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_model(db_session) .where(lambda d: d.is_active) )
With additional filtering:
dim = ( LXIndexDimension(Product, lambda p: p.sku) .from_model(session) .where(lambda p: p.stock_quantity > 0) )
Note
This method is mutually exclusive with from_data()
The actual database query happens during model solving, not at definition time
Filters applied via where() are evaluated in Python after the query
See also
lumix.utils.orm: ORM integration utilities
- get_instances()[source]
Retrieve and filter data instances for this dimension.
This method retrieves instances from the configured data source (either direct data or ORM query) and applies any filters specified via where(). This is typically called internally during model solving, not by user code.
- Return type:
- Returns:
List of model instances after filtering
- Raises:
ValueError – If no data source is configured (neither from_data() nor from_model() has been called)
Examples
The method is typically called internally, but can be used for inspection:
dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_data(all_drivers) .where(lambda d: d.is_active) ) # Get filtered instances active_drivers = dim.get_instances() print(f"Found {len(active_drivers)} active drivers")
Note
For ORM-based dimensions, this triggers the database query
Filters are applied in Python after data retrieval
Results are not cached; each call may return different results if data changes
- Implementation Details:
The method follows this logic: 1. If _data is set, use it as the data source 2. Otherwise, if _session is set, query via ORM using LXTypedQuery 3. Otherwise, raise ValueError 4. Apply filter_func if present
- where(predicate)[source]
Apply a filter predicate to this dimension.
This method adds a filter that determines which model instances from this dimension should be included. Only instances where the predicate returns True will be included in the dimension’s expansion.
The filter is applied within the dimension itself, before any cross-dimension filters (where_multi) are applied when using cartesian products.
- Parameters:
predicate (
Callable[[TypeVar(TModel)],bool]) – A function that takes a model instance and returns True if it should be included, False otherwise- Return type:
LXIndexDimension[TypeVar(TModel)]- Returns:
Self for method chaining
Examples
Simple filter:
dim = ( LXIndexDimension(Driver, lambda d: d.id) .from_data(drivers) .where(lambda d: d.is_active) )
Complex filter with multiple conditions:
dim = ( LXIndexDimension(Product, lambda p: p.sku) .from_data(products) .where(lambda p: p.in_stock and p.price > 0 and not p.discontinued) )
Filter with attribute check:
dim = ( LXIndexDimension(Route, lambda r: (r.origin, r.dest)) .from_data(routes) .where(lambda r: r.distance < 1000 and r.is_operational) )
Note
Filters are evaluated in Python after data retrieval
Multiple where() calls will override previous filters (not combine them)
For multi-dimensional filtering across dimensions, use where_multi() on the variable
See also
lumix.core.variables.LXVariable.where_multi(): Cross-dimension filteringget_instances(): Where filtering is applied
Key Attributes:
model_type: The Python class representing the data modelkey_func: Function to extract index keys from instancesfilter_func: Optional predicate to filter instanceslabel: Optional human-readable label
Key Methods:
from_data(data): Provide data instances directlyfrom_model(session): Query data from ORM sessionwhere(predicate): Apply per-dimension filteringget_instances(): Retrieve filtered instances
LXCartesianProduct¶
Cartesian product support for multi-dimensional model indexing.
This module provides the LXCartesianProduct class, which combines multiple index dimensions to create multi-dimensional index spaces. This enables variables and constraints indexed by tuples of data models (e.g., (Driver, Date, Shift)).
- Classes:
LXCartesianProduct: Type-safe cartesian product of multiple index dimensions
Example
Two-dimensional product:
from lumix import LXCartesianProduct, LXIndexDimension
product = LXCartesianProduct(
LXIndexDimension(Driver, lambda d: d.id).from_data(drivers),
LXIndexDimension(Date, lambda dt: dt.date).from_data(dates)
).where(lambda driver, date: driver.is_available(date))
Three-dimensional product:
product = (
LXCartesianProduct(
LXIndexDimension(Driver, lambda d: d.id).from_data(drivers),
LXIndexDimension(Date, lambda dt: dt.date).from_data(dates)
)
.add_dimension(LXIndexDimension(Shift, lambda s: s.id).from_data(shifts))
.where(lambda driver, date, shift: driver.can_work_shift(shift))
)
See also
LXIndexDimension: Individual dimensionsindexed_by_product: Uses cartesian productsDriver Scheduling Example (examples/02_driver_scheduling): Real-world usage
- class lumix.indexing.cartesian.LXCartesianProduct(dim1, dim2)[source]
Bases:
Generic[TModel1,TModel2]Type-safe cartesian product of multiple index dimensions.
LXCartesianProduct combines two or more index dimensions to create a multi-dimensional index space. This enables variables and constraints indexed by tuples of data models, such as (Driver, Date) or (Warehouse, Product, TimePeriod).
The cartesian product generates all combinations of instances across dimensions, with optional cross-dimension filtering to create sparse index spaces (only valid combinations).
This class is the foundation of LumiX’s multi-model indexing capability, which is one of the library’s most powerful features for complex scheduling, routing, and allocation problems.
- Type Parameters:
TModel1: The data model type for the first dimension TModel2: The data model type for the second dimension
- Parameters:
dim1 (LXIndexDimension[TModel1])
dim2 (LXIndexDimension[TModel2])
- dimensions
List of LXIndexDimension objects defining each dimension
- _cross_filter
Optional predicate for filtering combinations across dimensions
Examples
Basic two-dimensional product:
product = LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) )
With cross-dimension filtering:
product = LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) ).where(lambda driver, date: date not in driver.days_off)
Three-dimensional product:
product = ( LXCartesianProduct( LXIndexDimension(Warehouse, lambda w: w.id).from_data(warehouses), LXIndexDimension(Product, lambda p: p.sku).from_data(products) ) .add_dimension(LXIndexDimension(Month, lambda m: m.id).from_data(months)) .where(lambda w, p, m: w.stocks_product(p) and m.is_active) )
Used in variable definition:
from typing import Tuple assignment = ( LXVariable[Tuple[Driver, Date], int]("assignment") .binary() .indexed_by_product( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) ) .where_multi(lambda driver, date: driver.is_available(date)) )
See also
LXIndexDimension: Individual dimensionslumix.core.variables.LXVariable.indexed_by_product(): Uses this classDriver Scheduling Example (examples/02_driver_scheduling): Complete example
Note
The cartesian product follows lazy evaluation - combinations are generated only when needed during model solving
Cross-dimension filters (where()) are applied after per-dimension filters
For N dimensions, the product generates O(n1 × n2 × … × nN) combinations before filtering
- __init__(dim1, dim2)[source]
Initialize a cartesian product with two dimensions.
Creates a two-dimensional cartesian product. Additional dimensions can be added via add_dimension() for 3D, 4D, or higher-dimensional index spaces.
- Parameters:
dim1 (
LXIndexDimension[TypeVar(TModel1)]) – First index dimensiondim2 (
LXIndexDimension[TypeVar(TModel2)]) – Second index dimension
Examples
Basic initialization:
product = LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) )
With per-dimension filters:
product = LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id) .from_data(drivers) .where(lambda d: d.is_active), LXIndexDimension(Date, lambda dt: dt.date) .from_data(dates) .where(lambda dt: dt.is_weekday) )
- __deepcopy__(memo)[source]
Custom deepcopy that handles dimensions and cross-filter functions.
This method enables what-if analysis on cartesian products by: 1. Deep copying all index dimensions (with ORM data materialization) 2. Safely copying the cross-filter lambda function
- Parameters:
memo – Dictionary for tracking circular references during deepcopy
- Returns:
Deep copy of this cartesian product with all dependencies resolved
- add_dimension(dim)[source]
Add another dimension to create 3D or higher-dimensional indexing.
This method extends a two-dimensional product to three or more dimensions. Each additional dimension multiplies the number of combinations (before filtering).
- Parameters:
dim (
LXIndexDimension) – Additional index dimension to add- Return type:
Self- Returns:
Self for method chaining
Examples
Three-dimensional indexing:
product = ( LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) ) .add_dimension(LXIndexDimension(Shift, lambda s: s.id).from_data(shifts)) )
Four-dimensional indexing:
product = ( LXCartesianProduct( LXIndexDimension(Warehouse, lambda w: w.id).from_data(warehouses), LXIndexDimension(Product, lambda p: p.sku).from_data(products) ) .add_dimension(LXIndexDimension(Customer, lambda c: c.id).from_data(customers)) .add_dimension(LXIndexDimension(Month, lambda m: m.id).from_data(months)) )
Note
Dimensions can be added in any order
The cross-filter predicate (if set via where()) must match the number of dimensions
Each added dimension increases the computational complexity exponentially
- where(predicate)[source]
Apply cross-dimension filtering to the cartesian product.
This method adds a filter that operates across all dimensions simultaneously, allowing you to exclude invalid combinations based on relationships between the dimension models. This is essential for creating sparse index spaces.
The predicate function receives one instance from each dimension as arguments (in the order dimensions were added) and should return True for valid combinations.
This is different from per-dimension filters (applied via LXIndexDimension.where()), which filter within a single dimension before the cartesian product is formed.
- Parameters:
predicate (
Callable[[TypeVar(TModel1),TypeVar(TModel2)],bool]) – A function that takes one model instance from each dimension and returns True if the combination is valid, False otherwise- Return type:
Self- Returns:
Self for method chaining
Examples
Two-dimensional filtering:
product = LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) ).where(lambda driver, date: date not in driver.days_off)
Three-dimensional filtering:
product = ( LXCartesianProduct( LXIndexDimension(Driver, lambda d: d.id).from_data(drivers), LXIndexDimension(Date, lambda dt: dt.date).from_data(dates) ) .add_dimension(LXIndexDimension(Shift, lambda s: s.id).from_data(shifts)) .where(lambda driver, date, shift: driver.can_work_shift(shift) and date not in driver.days_off and shift.requires_certification <= driver.certifications ) )
Complex business logic:
product = LXCartesianProduct( LXIndexDimension(Warehouse, lambda w: w.id).from_data(warehouses), LXIndexDimension(Customer, lambda c: c.id).from_data(customers) ).where(lambda warehouse, customer: warehouse.region == customer.region and warehouse.can_ship_to(customer.zip_code) and customer.preferred_warehouses is None or warehouse.id in customer.preferred_warehouses )
Note
The predicate is called for every combination after per-dimension filters
Multiple where() calls override previous filters (not combine them)
For best performance, use per-dimension filters first, then cross-dimension filters
The function signature must match the number of dimensions
- Performance Tip:
Apply filters that reduce data size at the dimension level (via LXIndexDimension.where()) before applying cross-dimension filters here. This reduces the number of combinations that need to be evaluated.
See also
lumix.indexing.dimensions.LXIndexDimension.where(): Per-dimension filteringlumix.core.variables.LXVariable.where_multi(): Alternative syntax
Key Attributes:
dimensions: List of index dimensions_cross_filter: Optional cross-dimension filter predicate
Key Methods:
add_dimension(dim): Add another dimension (for 3D+)where(predicate): Apply cross-dimension filtering
Usage Examples¶
Single-Dimension Indexing¶
from lumix import LXVariable, LXIndexDimension
# Create dimension for products
product_dim = (
LXIndexDimension(Product, lambda p: p.sku)
.from_data(products)
.where(lambda p: p.in_stock)
)
# Create variable indexed by products
production = (
LXVariable[Product, float]("production")
.continuous()
.indexed_by(lambda p: p.sku)
.from_data(products)
.where(lambda p: p.in_stock)
)
Two-Dimensional Indexing¶
from lumix import LXVariable, LXIndexDimension
from typing import Tuple
# Create dimensions
driver_dim = LXIndexDimension(Driver, lambda d: d.id).from_data(drivers)
date_dim = LXIndexDimension(Date, lambda dt: dt.date).from_data(dates)
# Create multi-indexed variable with cross-dimension filter
assignment = (
LXVariable[Tuple[Driver, Date], int]("assignment")
.binary()
.indexed_by_product(driver_dim, date_dim)
.where_multi(lambda driver, date: driver.is_available(date))
)
# Access variable values
for driver in drivers:
for date in dates:
if driver.is_available(date):
print(f"assignment[{driver}, {date}] = {assignment[driver, date]}")
Three-Dimensional Indexing¶
from lumix import LXCartesianProduct, LXIndexDimension
# Create three dimensions
warehouse_dim = LXIndexDimension(Warehouse, lambda w: w.id).from_data(warehouses)
product_dim = LXIndexDimension(Product, lambda p: p.sku).from_data(products)
month_dim = LXIndexDimension(Month, lambda m: m.id).from_data(months)
# Create 3D cartesian product
inventory = (
LXVariable[Tuple[Warehouse, Product, Month], float]("inventory")
.continuous()
.indexed_by_product(warehouse_dim, product_dim, month_dim)
.where_multi(lambda w, p, m: w.stocks_product(p) and m.is_active)
)
Per-Dimension vs Cross-Dimension Filtering¶
from lumix import LXIndexDimension
# Per-dimension filter: applied within each dimension
active_drivers = (
LXIndexDimension(Driver, lambda d: d.id)
.from_data(drivers)
.where(lambda d: d.is_active and d.years_experience >= 2)
)
weekdays = (
LXIndexDimension(Date, lambda dt: dt.date)
.from_data(dates)
.where(lambda dt: dt.is_weekday)
)
# Cross-dimension filter: applied to combinations
assignment = (
LXVariable[Tuple[Driver, Date], int]("assignment")
.binary()
.indexed_by_product(active_drivers, weekdays)
.where_multi(lambda driver, date:
date not in driver.days_off and
driver.is_available(date)
)
)
ORM Integration¶
from lumix import LXIndexDimension
from sqlalchemy.orm import Session
# Use ORM session instead of direct data
driver_dim = (
LXIndexDimension(Driver, lambda d: d.id)
.from_model(db_session)
.where(lambda d: d.is_active)
)
# Data is queried lazily when needed
active_drivers = driver_dim.get_instances()
Sparse Indexing¶
from lumix import LXVariable, LXIndexDimension
from typing import Tuple
# Create route network with only valid connections
routes = (
LXVariable[Tuple[City, City], float]("flow")
.continuous()
.indexed_by_product(
LXIndexDimension(City, lambda c: c.id).from_data(cities),
LXIndexDimension(City, lambda c: c.id).from_data(cities)
)
.where_multi(lambda origin, dest:
origin != dest and # No self-loops
(origin, dest) in valid_routes # Only valid connections
)
)
# Only valid (origin, dest) pairs create variables
print(f"Created {len(routes.variables)} route variables")
Type Hints¶
The indexing module is fully type-annotated for IDE support:
from typing import Tuple, Callable, List
from lumix.indexing import LXIndexDimension, LXCartesianProduct
# Type-safe dimension creation
driver_dim: LXIndexDimension[Driver] = (
LXIndexDimension(Driver, lambda d: d.id)
.from_data(drivers)
)
# Type-safe cartesian product
product: LXCartesianProduct[Driver, Date] = LXCartesianProduct(
LXIndexDimension(Driver, lambda d: d.id).from_data(drivers),
LXIndexDimension(Date, lambda dt: dt.date).from_data(dates)
)
# Type-safe filter predicates
driver_filter: Callable[[Driver], bool] = lambda d: d.is_active
cross_filter: Callable[[Driver, Date], bool] = lambda d, dt: d.is_available(dt)
# Type-safe instance retrieval
instances: List[Driver] = driver_dim.get_instances()
See Also¶
Core Module API - Core module (uses indexing for variables/constraints)
Utils Module API - Utils module (ORM integration)
Solution Module API - Solution module (mapping indexed solutions)
Examples¶
Complete Examples:
Driver Scheduling (examples/02_driver_scheduling) - Comprehensive multi-dimensional example
Production Planning (examples/01_production_planning) - Single-dimension example
Network Flow - Multi-dimensional routing with sparse indexing