API and CLI Values Reference

This page is the compact reference for values you can pass to BlazeRules from the command line or Python.

Interfaces / parity

BlazeRules exposes the same engine through four boundaries. The Python SDK and the blazerules CLI share identical data-plane capabilities — the same rules, operators, and input formats — and differ only in where records come from.

Interface	Input boundary	Use it for
Python SDK	In-memory objects (NDJSON bytes, Arrow `RecordBatch`)	Embedding the engine in a Python service or notebook.
`blazerules` CLI	Files, `s3://` objects, stdin, and Kafka	Batch evaluation, validation, backtesting, and native Kafka streaming from the shell.
`blazerules_agent`	HTTP, tailed file, or stdin	Long-running local ingest with batching and de-dupe.
`blazerules_dashboard`	Read-only local UI	Inspecting rules, decision logs, dead-letter logs, and health.

blazerules_agent and blazerules_dashboard are separate binaries — they are not subcommands of blazerules (blazerules agent / blazerules dashboard intentionally error and point you to the standalone binaries).

📘
--input chooses the input type
blazerules_agent --input accepts stdin, file_tail, or http. --source is only a free-form metadata label written into wrapped log records and output; it is not an input selector.

`blazerules` CLI

Use blazerules for full native batch evaluation, validation, backtesting, and Kafka streaming.

Command	Purpose
`blazerules info`	Print version, SIMD backend, CPU features, and enabled IO/model features.
`blazerules --version`	Print the version string (e.g. `0.5.4`) and exit. Also on `blazerules_agent` and `blazerules_dashboard`.
`blazerules validate`	Load and validate a rule file.
`blazerules eval`	Evaluate a file, S3 object, or stdin batch.
`blazerules backtest`	Compare two rulesets over Parquet inputs.
`blazerules stream kafka`	Run a native Kafka microbatch consume/evaluate/produce loop.

`blazerules validate`

Argument	Values	Default	Meaning
`--rules`	path or `s3://bucket/key`	required	Rule YAML to load and validate.
`--model`	`name=path`, repeatable	none	Register ONNX model so `model_score` rules validate.
`--sample`	path to NDJSON	none	Optionally push a sample NDJSON batch through the engine; output JSON then includes `sample_records` and `sample_skipped`.
`--config`	path	none	Unified single-run config; explicit flags override it.

`blazerules eval`

Argument	Values	Default	Meaning
`--rules`	path or `s3://bucket/key`	required	Rule YAML.
`--input` / `--format`	`ndjson`, `jsonl`, `json`, `json-array`, `debezium`, `arrow-ipc`, `arrow`, `parquet`, `csv`, `avro`, `protobuf`, `protobuf-delimited`, `auto`	`auto`	Input format. `jsonl` is an alias of `ndjson`; `arrow` is an alias of `arrow-ipc`; `json-array` evaluates a top-level JSON array directly instead of converting it to NDJSON first. `avro` auto-detects an Object Container File (multi-record, self-describing schema) via magic bytes, falling back to one bare Avro value otherwise. `protobuf` decodes exactly one bare message; `protobuf-delimited` reads a file of N varint-length-prefixed messages instead — not auto-detected, since Protobuf has no magic bytes to tell the two apart.
`--path`	path or `s3://bucket/key`, repeatable	required unless `--stdin`	Input object(s).
`--stdin`	flag	false	Read NDJSON/JSON bytes from stdin.
`--batch-size`	integer	`10000`	Batch size for batch/file formats.
`--threads`	integer	auto	Worker thread count for evaluation. `0`/unset means automatic.
`--mmap`	flag	false	`ndjson`/`jsonl` local files only: mmap the whole file zero-copy instead of streaming bounded chunks. Faster for large files with `--threads` > 2, since the default streaming chunk size is memory-bounded rather than width-unbounded; trades away that memory bound.
`--schema`	path or `s3://bucket/key`	required for a bare (non-OCF) Avro file	Avro schema JSON. Not needed for an Object Container File — it carries its own schema.
`--descriptor`	path or `s3://bucket/key`	required for `protobuf`/`protobuf-delimited`	Serialized protobuf descriptor set.
`--message`	fully-qualified type	required for `protobuf`/`protobuf-delimited`	Message type inside the descriptor set.
`--op-field`	field name	`__op`	Debezium operation field.
`--model`	`name=path`, repeatable	none	Register ONNX model (local path or `s3://`) for `model_score` rules.
`--output`	`none`, `summary`, `decisions-jsonl`, `grouped-decisions`, `rule-counts`, `bitmasks`, `arrow-ipc`	`summary` (or `decisions-jsonl` when `--output-path` is set)	Output shape.
`--output-detail`	`counts`, `codes`, `decisions`, `bitmasks`	`decisions`	Result materialization level (see Decisions & Scoring).
`--output-path`	path	stdout	Output file.
`--decision-log`	path	empty	Engine decision log path.
`--dead-letter-log`	path	empty	Engine DLQ path.
`--ingest-error-mode`	`skip_and_count`, `skip_to_dead_letter`, `hard_fail`	`skip_and_count`	Bad-record policy.
`--type-mismatch-mode`	`null_on_type_error`, `coerce`, `hard_fail_type`	`null_on_type_error`	Type-drift policy.
`--simd-backend`	`auto`, `scalar`, `neon`, `avx2`, `avx512`	`auto`	SIMD backend override.
`--aws-profile`	profile name	environment	AWS profile for `s3://` resources.
`--aws-region`	region	environment	AWS region.
`--aws-endpoint-url`	URL	environment	Custom S3 endpoint.
`--config`	path	none	Unified single-run config; explicit flags override values from it.

blazerules eval --rules rules.yaml --input ndjson --path events.ndjson --output grouped-decisions
blazerules eval --rules rules.yaml --input arrow-ipc --path events.arrow --output summary
blazerules eval --rules rules.yaml --input protobuf --path event.pb --descriptor schema.desc --message payments.Transaction
blazerules eval --rules rules.yaml --input avro --path events.avro --output summary
blazerules eval --rules rules.yaml --input protobuf-delimited --path events.pb --descriptor schema.desc --message payments.Transaction

`blazerules stream kafka`

Argument	Values	Default	Meaning
`--rules`	path or `s3://bucket/key`	required	Rule YAML.
`--brokers`	broker list	required	Kafka bootstrap servers.
`--group-id`	string	`blazerules`	Consumer group.
`--input-topic`	topic or comma list	required	Input topic(s). Alias: `--input-topics`.
`--output-topic`	topic	empty	Decision output topic. Empty disables produce.
`--output-mode`	`rows`, `grouped`, `none`	`rows`	Shape of records written to `--output-topic`: one row per decision, per-decision-label counts, or no output.
`--dlq-topic`	topic	empty	Route records that fail to decode to a dead-letter topic and keep consuming instead of aborting.
`--format`	`json`, `ndjson`, `debezium`, `arrow-ipc`, `avro`, `protobuf`	`json`	Kafka message payload format.
`--arrow-validation`	`full`, `structural`, `trusted`	`structural`	Validation level applied to Arrow IPC Kafka payloads before decoding.
`--op-field`	field name	`__op`	Debezium operation field.
`--schema`	path or `s3://bucket/key`	required for Avro	Avro schema JSON.
`--descriptor`	path or `s3://bucket/key`	required for Protobuf	Serialized protobuf descriptor set.
`--message`	fully-qualified type	required for Protobuf	Message type inside descriptor set.
`--model`	`name=path`, repeatable	none	Register ONNX model for `model_score` rules.
`--batch-size`	integer	`2048`	Kafka microbatch size.
`--workers`	integer	`1`	Concurrent evaluation worker threads, each with its own `RuleEngine` shard. Pure CPU work; no Kafka calls happen on these threads.
`--queue-depth`	integer	`64`	Depth of every bounded queue in the pipeline: each worker's input queue and the shared delivery queue.
`--poll-timeout-ms`	integer	`1000`	Poll timeout.
`--flush-timeout-ms`	integer	`5000`	Producer flush timeout.
`--flush-interval-ms`	integer	`250`	How often the delivery thread batches and flushes produced messages to Kafka. Offsets commit only after a successful flush.
`--max-messages`	integer	`0`	`0` means unlimited.
`--max-batches`	integer	`0`	`0` means unlimited.
`--commit-offsets`	bool	`true`	Commit after successful evaluation/output.
`--partition-affine`	bool	`true`	Route every message for a `(topic,partition)` to the same worker, preserving per-partition order. Forced `true` whenever `--commit-offsets` is enabled; optional (round-robin dispatch) only when `--commit-offsets=false`.
`--consumer-conf`	`k=v`, repeatable	none	Arbitrary librdkafka consumer settings, e.g. `security.protocol=SASL_SSL`.
`--producer-conf`	`k=v`, repeatable	none	Arbitrary librdkafka producer settings, e.g. `sasl.mechanism=PLAIN`.
`--config`	path	none	Unified single-run config; explicit flags override it.

The output JSON reports batches, messages, matched, emitted, eval_ms, delivery_errors, and dlq_routed (records routed to --dlq-topic).

blazerules stream kafka \
  --rules rules.yaml \
  --brokers localhost:9092 \
  --input-topic transactions \
  --output-topic decisions \
  --format avro \
  --schema transaction.avsc

`blazerules_agent` CLI

Use --config agent.yaml for multiple instances, or pass single-instance flags directly.

Argument	Values	Default	Meaning
`--config`	path or `s3://bucket/key`	none	Load one or more `instances:` from YAML.
`--name`	string	`default`	Instance label.
`--rules`	path or `s3://bucket/key`	required	Rule YAML for this instance.
`--input`	`stdin`, `file_tail`, `http`	`stdin`	Input adapter.
`--path`	path	none	File path for `file_tail`.
`--host`	IP or hostname	`127.0.0.1`	HTTP bind host.
`--port`	integer	`9480`	HTTP bind port.
`--batch-size`	integer	`2048`	Records buffered before a size flush.
`--flush-ms`	integer	`1000`	Milliseconds before flushing a partial batch.
`--eval-shards`	integer	`1`	Pool of cloned engines for parallel evaluation of stateless rulesets; auto-downgrades to 1 if dedupe is on or the ruleset uses windows / an unbound schema.
`--http-threads`	integer	`max(4, cores)`	httplib's own accept/read thread pool size; governs concurrent connection handling, not evaluation or write concurrency.
`--http-queue-depth`	integer	`256`	Depth of httplib's own request queue feeding `--http-threads`; a full queue closes the connection immediately with no HTTP response.
`--eval-queue-depth`	integer	`64`	Depth of the bounded queue feeding eval-worker threads; a full queue returns HTTP 429.
`--sink-queue-depth`	integer	`64`	Depth of the bounded queue feeding sink-worker threads; a full queue returns HTTP 503.
`--sink-workers`	integer	`1`	Threads that dequeue evaluated batches and write them to the decision log; parallelizes Arrow encoding, not just the final write.
`--max-request-mb`	integer	`256`	Hard cap on one HTTP request body, in MiB; larger requests get HTTP 413.
`--ack-mode`	`durable`, `evaluated`	`durable`	`durable` acks after the sink worker writes the decision; `evaluated` acks right after evaluation, before the write.
`--output`	`stdout`, `ndjson`	`stdout`	Decision output sink.
`--output-path`	path	none	NDJSON decision path when `--output ndjson`.
`--service`	string	`--name`	Metadata label.
`--source`	string	`log`	Metadata label, not an input selector.
`--dedupe-key`	field name, repeatable	none	Enables local de-dupe using one or more fields.
`--dedupe-ttl-seconds`	integer	`86400`	How long duplicate keys stay suppressed.
`--aws-region`	region	environment	AWS region for `s3://` rules/models/output (else `AWS_REGION`).
`--aws-endpoint-url`	URL	environment	Custom S3 endpoint (else `AWS_ENDPOINT_URL`).
`--version`	flag	—	Print the version string and exit.

blazerules_agent \
  --rules rules.yaml \
  --input http \
  --host 127.0.0.1 \
  --port 9480 \
  --batch-size 4096 \
  --flush-ms 50 \
  --output ndjson \
  --output-path decisions.ndjson \
  --service payments-api \
  --source http-json \
  --dedupe-key event_id

Agent `instances:` YAML

The same settings can be placed in YAML. A top-level instances: list starts one thread per instance.

YAML key	Values	Default	Meaning
`instances[].name`	string	`default`	Instance label.
`instances[].rules`	path or `s3://bucket/key`	required	Rule YAML.
`instances[].batch_size`	integer	`2048`	Size flush threshold.
`instances[].flush_ms`	integer	`1000`	Time flush threshold.
`instances[].eval_shards`	integer	`1`	Parallel clone-engine pool for stateless rulesets (auto-downgrades with dedupe, windows, or an unbound schema).
`instances[].http_threads`	integer	`max(4, cores)`	httplib's own accept/read thread pool size.
`instances[].http_queue_depth`	integer	`256`	Depth of httplib's own request queue; a full queue closes the connection immediately.
`instances[].eval_queue_depth`	integer	`64`	Depth of the bounded queue feeding eval-worker threads; a full queue returns HTTP 429.
`instances[].sink_queue_depth`	integer	`64`	Depth of the bounded queue feeding sink-worker threads; a full queue returns HTTP 503.
`instances[].sink_workers`	integer	`1`	Threads that write evaluated batches to the decision log.
`instances[].max_request_mb`	integer	`256`	Hard cap on one HTTP request body, in MiB; larger requests get HTTP 413.
`instances[].ack_mode`	`durable`, `evaluated`	`durable`	Acknowledge after the sink write (`durable`) or right after evaluation (`evaluated`).
`instances[].service`	string	instance name	Service metadata label.
`instances[].source`	string	`log`	Source metadata label.
`instances[].input.type`	`stdin`, `file_tail`, `http`	`stdin`	Input adapter.
`instances[].input.path`	path	none	Required for `file_tail`.
`instances[].input.host`	host	`127.0.0.1`	HTTP bind host.
`instances[].input.port`	integer	`9480`	HTTP bind port.
`instances[].output.type`	`stdout`, `ndjson`	`stdout`	Decision sink.
`instances[].output.path`	path	none	Required for `ndjson`.
`instances[].dedupe.enabled`	`true`, `false`	`false`	Enable local duplicate suppression.
`instances[].dedupe.key_fields`	list of field names	none	Duplicate key fields.
`instances[].dedupe.ttl_seconds`	integer	`86400`	Duplicate suppression TTL.

`blazerules_dashboard` CLI

Argument	Values	Default	Meaning
`--host`	IP or hostname	`127.0.0.1`	HTTP bind host.
`--port`	integer	`9470`	Dashboard port.
`--poll-ms`	integer	`1000`	Poll interval for watched files and metrics.
`--tail-lines`	integer	`5000`	Bounded recent-line buffer per log file.
`--decision-log`	path	none	Compact decision NDJSON to inspect.
`--decision-log-dir`	path or `s3://bucket/prefix/`	none	Directory of per-instance decision logs (NDJSON/Arrow), merged into one view; use instead of `--decision-log` for a multi-instance agent.
`--dead-letter-log`	path	none	Dead-letter NDJSON to inspect.
`--metrics-url`	URL	none	Prometheus exposition URL.
`--results-jsonl`	path	none	Stress matrix benchmark JSONL.
`--rules`	path	none	Active rules YAML for validation and visualizer.
`--rules-dir`	path or `s3://bucket/prefix/`	none	Directory of ruleset YAMLs; the Ruleset selector lists each `.yaml`/`.yml` stem and the visualizer scopes to the one matching the selected instance.
`--candidate-rules`	path	none	Candidate rules YAML for diff/validation.
`--rules-history-dir`	path	none	Directory of previous YAML versions.
`--aws-region`	region	environment	AWS region for `s3://` sources (else `AWS_REGION`).
`--aws-endpoint-url`	URL	environment	Custom S3 endpoint (else `AWS_ENDPOINT_URL`).
`--version`	flag	—	Print the version string and exit.

Python module constants and enums

Name	Values
`blazerules.ColumnType`	`FLOAT32`, `FLOAT64`, `INT32`, `INT64`, `CATEGORICAL`, `ENTITY_KEY`, `TIMESTAMP_MS`, `BOOLEAN`, `STRING`
`blazerules.ActionType`	`APPROVE`, `FLAG`, `REVIEW`, `BLOCK`, `SCORE`
`blazerules.RuleFileFormat`	`YAML`, `JSON`
`blazerules.OutputDetail`	`COUNTS`, `CODES`, `DECISIONS`, `BITMASKS`
`blazerules.IngestErrorMode`	`SKIP_AND_COUNT`, `SKIP_TO_DEAD_LETTER`, `HARD_FAIL`
`blazerules.TypeMismatchMode`	`NULL_ON_TYPE_ERROR`, `COERCE`, `HARD_FAIL_TYPE`
`blazerules.SimdBackend`	`SCALAR`, `NEON`, `SSE2`, `AVX2`, `AVX512`
`blazerules.TraceMode`	`TRACE_NONE`, `TRACE_SAMPLED`, `TRACE_ALL`

`EngineConfig`

Attribute	Values	Default	Meaning
`batch_size`	integer	`10000`	Preferred internal batch size.
`parallel_threshold`	integer	`1000`	Row threshold before parallel execution is considered.
`eval_thread_count`	integer	`0`	`0` means automatic worker count.
`output_detail`	`OutputDetail.COUNTS`, `OutputDetail.CODES`, `OutputDetail.DECISIONS`, `OutputDetail.BITMASKS`	`OutputDetail.BITMASKS`	How much result data to materialize. Use `COUNTS` for aggregate-only runs, `CODES` for compact routing, `DECISIONS` for normal per-row decisions, and `BITMASKS` for per-rule masks.
`ingest_error_mode`	`SKIP_AND_COUNT`, `SKIP_TO_DEAD_LETTER`, `HARD_FAIL`	`SKIP_AND_COUNT`	Bad-record policy.
`type_mismatch_mode`	`NULL_ON_TYPE_ERROR`, `COERCE`, `HARD_FAIL_TYPE`	`NULL_ON_TYPE_ERROR`	Type drift policy after schema binding.
`max_error_samples`	integer	`16`	Max bad-record samples kept in `BatchResult`.
`decision_log_path`	string path	empty	Empty disables engine-level decision logging.
`dead_letter_path`	string path	empty	Dead-letter output path for skipped records.
`max_window_entities`	integer	`10000000`	Window state capacity hint.
`arena_size_bytes`	integer	`8388608`	Per-worker arena size.
`max_dict_size_per_column`	integer	`100000`	Dictionary cap for non-entity categoricals.
`enable_selection_vectors`	bool	`true`	Use survivor-index execution for selective predicates.
`selection_vector_threshold`	float	`0.20`	Survivor fraction below which index execution becomes useful.
`enable_adaptive_predicate_ordering`	bool	`true`	Order cheap/high-rejection predicates first.
`enable_no_validity_fast_path`	bool	`true`	Skip validity bitmap work for known non-null columns.
`enable_prefetch`	bool	`false`	Enable explicit prefetch in long scans.
`enable_thread_affinity`	bool	`false`	Best-effort worker affinity.
`result_buffer_reuse`	bool	`true`	Reuse output buffers across evaluations.
`simd_backend_override`	`auto`, `scalar`, `neon`, `sse2`, `avx2`, `avx512`	`auto`	Runtime SIMD backend override.
`enable_avx512`	bool	`false`	Make AVX-512 eligible during auto selection.
`hot_reload_poll_seconds`	integer	`5`	Default hot-reload polling interval.
`hot_reload_validate_conflicts`	bool	`true`	Analyze conflicts before hot-reload activation.
`hot_reload_keep_previous_on_failure`	bool	`true`	Keep old rules active when reload fails.
`trace_sample_rate`	float	`0.05`	Trace sampling rate used when `trace_mode` is `TraceMode.TRACE_SAMPLED`.
`trace_mode`	`TraceMode.TRACE_NONE`, `.TRACE_SAMPLED`, `.TRACE_ALL`	`TraceMode.TRACE_NONE`	Enables/disables tracing; `trace_sample_rate` only has an effect under `TRACE_SAMPLED`.
`eviction_sweep_interval_minutes`	float	`5`	How often window/dedupe state eviction sweeps run, in minutes.

`RuleEngine` methods

Method	Main arguments	Returns / effect
`RuleEngine()`	optional `EngineConfig`; optional explicit schema	Creates an engine. Omit schema to infer from the first batch.
`load_rules(path)`	local path or `s3://...`	Loads, validates, compiles, and activates rules.
`load_rules_from_string(text, format=YAML)`	YAML/JSON text	Compiles rules from memory.
`reload_rules_now(path)`	local path or `s3://...`	Compile-and-swap immediately if valid.
`enable_hot_reload(path, poll_interval_seconds=5)`	path and polling seconds	Starts background reload checks.
`stop_hot_reload()`	none	Stops the hot-reload thread.
`hot_reload_status()`	none	Returns `HotReloadStatus`.
`evaluate_ndjson(payload)`	bytes-like NDJSON	Evaluates one JSON batch.
`evaluate_ndjson_padded(payload, logical_size)`	padded bytes-like NDJSON	Evaluates already padded NDJSON without copying the padding.
`evaluate_json_array(payload)`	bytes-like top-level JSON array	Evaluates one JSON array batch without converting it to NDJSON.
`evaluate_json_array_padded(payload, logical_size)`	padded bytes-like top-level JSON array	Evaluates an already padded JSON array batch.
`evaluate_messages(messages)`	sequence of JSON strings/bytes	Convenience path for smaller integrations.
`evaluate_batch(batch, validate=True)`	`pyarrow.RecordBatch`, optional validation skip	Evaluates typed Arrow data. Import through Arrow's C Data Interface is zero-copy either way; `validate=True` (default) additionally runs a full `ValidateFull()` scan on the imported batch, which callers who've already validated the same data upstream can skip with `validate=False`.
`evaluate_ndjson_file(path)`	local path or `s3://...`	Reads/evaluates NDJSON file.
`create_shards(shard_count)`	integer	Returns partition-affine engine shards.
`evaluate_partition_messages(partition_id, messages)`	partition id, JSON messages	Routes to partition-local state.
`evaluate_partition_ndjson_padded(partition_id, payload, logical_size)`	partition id, payload	Partition-aware NDJSON evaluation.
`evaluate_partition_batch(partition_id, batch, validate=True)`	partition id, Arrow batch, optional validation skip	Partition-aware Arrow evaluation; same `validate` behavior as `evaluate_batch`.
`register_model(name, path)`	model name, ONNX path or `s3://...`	Registers a model used by `model_score`.
`reset_window_state()`	none	Clears in-memory window state.
`backtest(parquet_path, rules_a, rules_b, label_column=None)`	path(s), two rule sets	Runs offline comparison.
`analyze_conflicts(rules_path)`	rules path	Returns `ConflictReport`.
`enable_metrics()` / `reset_metrics()` / `metrics_snapshot()`	none	In-process metrics.
`stats()`	none	Returns `EngineStats` (`batches_evaluated`, `records_evaluated`, `records_skipped`).

`BatchResult`

Attribute or method	Meaning
`n_records`, `n_matched`	Batch size and matched-row count.
`decisions`, `decision_codes`, `decision_label_map`	Per-row decision labels or compact integer codes.
`scores`, `risk_bands`, `winning_rule_ids`	Per-row scoring outputs.
`match_counts`	Rule ID to fire count.
`matched_indices`	NumPy array of all matched row indices.
`indices_for_decision(label)`	NumPy indices for one decision label.
`indices_for_not_decision(label)`	NumPy indices for all other labels.
`grouped_decision_indices()`	Dict of decision label to NumPy index array.
`grouped_winning_rule_indices()`	Dict of winning rule id to NumPy index array.
`indices_for_rule(rule_id)`	NumPy indices for a rule; requires rule bitmasks.
`result[rule_id]`	Boolean NumPy mask; requires `OutputDetail.BITMASKS`.
`decision_codes_buffer()`, `matched_indices_buffer()`, `rule_bitmask_buffer(rule_id)`	Zero-copy memoryview buffers for high-throughput routing.
`messages_processed`, `messages_skipped`, `error_counts`, `error_samples`, `last_ingest_error`	Ingest diagnostics.
`timing`	Dict of internal timing fields in milliseconds.

`EngineStats`

Returned by RuleEngine.stats() — cumulative counters since the engine was created, not per-batch.

Attribute	Meaning
`batches_evaluated`	Number of `evaluate_`/`evaluate_partition_` calls handled so far.
`records_evaluated`	Total records that reached rule evaluation.
`records_skipped`	Total records dropped by ingest error handling before evaluation.

`blazerules_io`

Name	Values / arguments	Meaning
`has_kafka`, `has_avro`, `has_protobuf`	bool	Runtime capability flags for optional IO features.
`FileFormat`	`AUTO`, `ARROW_IPC`, `PARQUET`, `CSV`, `NDJSON`	File reader format selector.
`ArrowIpcValidationLevel`	`FULL`, `STRUCTURAL`, `TRUSTED`	Arrow IPC batch validation strictness, used by `FileReadOptions.arrow_validation`.
`read_ndjson_bytes(path)`	local path or `s3://...`	Reads exact NDJSON bytes. `s3://` paths stream natively off Arrow's S3 filesystem by default, falling back to an AWS-CLI download if nothing has been read yet; takes no options, so it always runs with `FileReadOptions` defaults.
`read_record_batches(path, format="auto", batch_size=65536)`	local/S3 path	Reads Arrow IPC, Parquet, CSV, or NDJSON into a list of Arrow batches. Thin wrapper over `for_each_record_batch` that materializes every batch and does not accept a `FileReadOptions`, so it also always runs with defaults.
`FileReadOptions`	see below	Options for `for_each_record_batch`: native S3 mode, column projection, validation level.
`for_each_record_batch(path, format, callback, options=FileReadOptions())`	local/S3 path, `callback(batch) -> bool \| None`	Streams Arrow IPC/Parquet/CSV/NDJSON one Arrow batch at a time instead of materializing the file; return `False` from `callback` to stop early. The only Python entry point that accepts `FileReadOptions`.
`ArrowIpcDecoder.decode_batch(frames)`	sequence of frames	Decodes binary Arrow IPC to one Arrow batch.
`AvroDecoder(schema_json).decode_batch(frames)`	sequence of Avro records	Decodes one Avro record per bare frame to Arrow when Avro support is built.
`looks_like_avro_ocf(data)`	bytes	`True` if `data` starts with the Avro Object Container File magic bytes (`"Obj\x01"`) — safe to auto-detect, unlike Protobuf framing.
`decode_avro_ocf_file_each(path, callback, batch_size=10000)`	local path, `callback(batch) -> bool \| None`, batch size	Reads a real multi-record Avro OCF file (the format Spark/Hadoop/Kafka Connect produce) using its own embedded schema — no `AvroDecoder`/schema needed — invoking `callback` once per batch of up to `batch_size` records; return `False` to stop early. Returns total records processed. Same capability as CLI `--input avro` on an OCF file.
`ProtobufDecoder(descriptor_set_bytes, message_type).decode_batch(frames)`	sequence of protobuf records	Decodes one protobuf record per bare frame to Arrow when Protobuf support is built.
`ProtobufDecoder.decode_delimited_file_each(path, callback, batch_size=10000)`	local path, `callback(batch) -> bool \| None`, batch size	Reads a file of N varint-length-delimited Protobuf messages, invoking `callback` once per batch. Same capability as CLI `--input protobuf-delimited`.
`ProtobufDecoder.decode_delimited_file_parallel(path, callback, batch_size=10000, worker_count=1)`	as above, plus worker thread count	Same as `decode_delimited_file_each`, but parses `batch_size`-sized chunks concurrently across `worker_count` threads — callback delivery order is unaffected, only the CPU-heavy parse step runs in parallel.
`unwrap_debezium(messages, op_field="__op")`	JSON CDC messages	Converts Debezium envelopes to NDJSON bytes.
`KafkaConsumer(brokers, group_id, topics, conf={})`	Kafka connection settings	Polls JSON batches or raw records with metadata.
`KafkaProducer(brokers, conf={})`	Kafka connection settings	Produces decision events.
`StreamRunConfig`	see below	C++-owned Kafka consume/evaluate/produce loop config.
`run_stream(engine, config)`	`RuleEngine`, `StreamRunConfig`	Runs the native Kafka microbatch loop for JSON/NDJSON, Debezium, Arrow IPC, Avro, or Protobuf payloads.

StreamRunConfig fields:

Attribute	Type	Meaning
`brokers`	string	Kafka bootstrap servers.
`group_id`	string	Consumer group.
`input_topics`	list of strings	Topics to consume.
`output_topic`	string	Topic for compact decisions.
`output_mode`	`rows`, `grouped`, `none`	Shape of records written to `output_topic`: one row per decision, per-decision-label counts, or no output.
`dlq_topic`	string	Dead-letter topic for records that fail to decode; empty disables DLQ routing.
`consumer_conf`, `producer_conf`	dict string to string	Extra librdkafka settings (SASL/SSL, etc.).
`batch_size`	integer	Max records per engine evaluation.
`worker_count`	integer	Concurrent evaluation worker threads, each with its own `RuleEngine` shard.
`queue_depth`	integer	Depth of every bounded queue in the pipeline: each worker's input queue and the shared delivery queue.
`poll_timeout_ms`	integer	Poll wait.
`flush_timeout_ms`	integer	Producer flush wait.
`flush_interval_ms`	integer	How often the delivery thread batches and flushes to Kafka; offsets commit only after a successful flush.
`max_messages`, `max_batches`	integer	Local run limits; leave at zero for unbounded.
`commit_offsets`	bool	Commit offsets after a batch is processed.
`partition_affine`	bool	Route every message for a `(topic, partition)` to the same worker, preserving order; forced `true` whenever `commit_offsets` is `true`.
`payload_format`	`json`, `ndjson`, `debezium`, `arrow-ipc`, `avro`, `protobuf`	Kafka message payload format.
`arrow_validation`	`ArrowIpcValidationLevel.FULL`, `.STRUCTURAL`, `.TRUSTED`	Validation level applied to Arrow IPC Kafka payloads before decoding.
`avro_schema_json`	string	Required when `payload_format == "avro"`.
`protobuf_descriptor_set`	bytes/string	Required when `payload_format == "protobuf"`.
`protobuf_message_type`	string	Required when `payload_format == "protobuf"`.
`debezium_op_field`	string	Operation field injected for Debezium unwrap.

FileReadOptions fields:

Attribute	Type	Default	Meaning
`batch_size`	integer	`65536`	Rows requested per Arrow batch from Parquet/CSV/Arrow-IPC readers.
`ndjson_chunk_bytes`	integer	`8388608`	NDJSON chunk size in bytes; used by the C++ `for_each_ndjson_chunk` visitor, which is not currently exposed to Python.
`included_fields`	list of strings	`[]` (all fields)	Column projection by field name, pushed into Parquet row-group reads and CSV's column parser. Ignored by Arrow IPC (see `included_field_indices`) and by NDJSON.
`included_field_indices`	list of integers	`[]` (all fields)	Column projection by index, for Arrow IPC only. Ignored by Parquet, CSV, and NDJSON.
`use_threads`	bool	`true`	Use Arrow's internal multithreading for the read.
`native_s3`	bool	`true`	Stream `s3://` reads directly through Arrow's `S3FileSystem` instead of downloading the object via the AWS CLI first.
`allow_s3_cli_fallback`	bool	`true`	If a native S3 read fails before any batch has reached the caller, retry the same object through the AWS-CLI-download path. Has no effect once delivery has started — no risk of a duplicate delivery.
`arrow_validation`	`ArrowIpcValidationLevel`	`FULL`	Batch validation strictness. Only the Arrow IPC reader path applies this today; Parquet, CSV, and NDJSON reads always run at the least-strict (`TRUSTED`) level regardless of this setting.

📘
native_s3 needs an S3-enabled build
Native streaming is compiled in via the BLAZERULES_IO_S3 CMake option (default ON). Unlike has_kafka/has_avro/has_protobuf, there's no runtime capability flag for it — on a build without it, native_s3=True silently falls through to the AWS-CLI path when allow_s3_cli_fallback is True (the default), or raises when it is False.

🚧
arrow_validation is Arrow-IPC-only today
Setting arrow_validation to STRUCTURAL or TRUSTED only changes behavior for Arrow IPC files. Parquet, CSV, and NDJSON reads currently always validate at the least-strict level no matter what arrow_validation is set to.

AWS and S3 helpers

Python function	Meaning
`set_aws_profile(profile, clear_env_credentials=True)`	Use an AWS CLI profile and optionally ignore stale key env vars.
`clear_aws_profile()` / `current_aws_profile()`	Clear or inspect profile selection.
`set_aws_region(region)`	Set region for `s3://` reads.
`clear_aws_region()` / `current_aws_region()`	Clear or inspect region selection.
`set_aws_endpoint_url(endpoint_url)`	Set S3-compatible endpoint for MinIO, R2, or LocalStack.
`clear_aws_endpoint_url()` / `current_aws_endpoint_url()`	Clear or inspect endpoint.
`set_aws_credentials(access_key_id, secret_access_key, session_token="", region="")`	Set process credentials for short-lived local scripts.
`clear_aws_credentials()`	Remove process AWS key variables.

Equivalent environment variables:

export BLAZERULES_AWS_PROFILE=personal
export BLAZERULES_AWS_REGION=us-east-1
export BLAZERULES_AWS_ENDPOINT_URL=http://127.0.0.1:9000

Rule values

The rule operator list is maintained in Operator Reference. The production shape of a full rule file is in Production YAML Guide.

API and CLI Values Reference

Interfaces / parity

`--input` chooses the input type

`blazerules` CLI

`blazerules validate`

`blazerules eval`

`blazerules stream kafka`

`blazerules_agent` CLI

Agent `instances:` YAML

`blazerules_dashboard` CLI

Python module constants and enums

`EngineConfig`

`RuleEngine` methods

`BatchResult`

`EngineStats`

`blazerules_io`

`native_s3` needs an S3-enabled build

`arrow_validation` is Arrow-IPC-only today

AWS and S3 helpers

Rule values

Interfaces / parity

--input chooses the input type

blazerules CLI

blazerules validate

blazerules eval

blazerules stream kafka

blazerules_agent CLI

Agent instances: YAML

blazerules_dashboard CLI

Python module constants and enums

EngineConfig

RuleEngine methods

BatchResult

EngineStats

blazerules_io

native_s3 needs an S3-enabled build

arrow_validation is Arrow-IPC-only today

AWS and S3 helpers

Rule values

`--input` chooses the input type

`blazerules` CLI

`blazerules validate`

`blazerules eval`

`blazerules stream kafka`

`blazerules_agent` CLI

Agent `instances:` YAML

`blazerules_dashboard` CLI

`EngineConfig`

`RuleEngine` methods

`BatchResult`

`EngineStats`

`blazerules_io`

`native_s3` needs an S3-enabled build

`arrow_validation` is Arrow-IPC-only today