API Reference¶

questdb¶

questdb.connect(conf_str: str | None = None, *, host: str | None = None, port: int | str | None = None, tls: bool | None = None, connection_listener: Callable[[ConnectionEvent], None] | None = None, connection_event_inbox_capacity: int = 0, error_handler: Callable[[SenderError], None] | None = None, error_event_inbox_capacity: int = 0, **params) → QuestDB[source]¶

Connect to a QuestDB deployment and return a QuestDB handle.

This is the deployment-level entry point and the default for new code. The handle owns connection pools for both ingestion and queries: borrow row-building senders with QuestDB.sender(), bulk-load DataFrames with QuestDB.dataframe(), run SQL with QuestDB.query(), and borrow reader leases with QuestDB.reader(). The standalone Sender is the complementary connection-level API (one sender = one connection) for point-to-point needs: ILP/HTTP transactions, ILP/TCP, QWP/UDP datagrams, and manually driven single ws connections.

Pass either a QWP/WebSocket configuration string (ws::addr=host:port; or wss::...) or the equivalent keywords: host, port (default 9000) and tls (default False) select the endpoint. Either way, any further keyword arguments are added as configuration-string settings (username='u', sender_pool_max=4, …; booleans normally map to on/off, while tls_verify=False maps to unsafe_off); a setting present both in the string and as a keyword raises ValueError, matching Sender.from_conf(). One configuration addresses the whole deployment; list every cluster node in a single addr server list.

By default connect() opens the warm minimums up front and fails fast when the server is unreachable or rejects the credentials. Set lazy_connect=True (or lazy_connect=true in the string) to tolerate a down server at startup: nothing connects here, sender leases buffer locally and connect in the background, and readers connect on the first query.

Pooled row senders auto-flush by default at 1,000 rows, 100 milliseconds, or a cap-derived byte threshold. Set auto_flush_bytes='off' to disable only the byte trigger, or auto_flush='off' to disable all automatic publishing. Auto-triggered publishes do not wait for server acknowledgement; use an explicit flush(wait=True) or wait() as an acknowledgement barrier.

connection_listener receives one ConnectionEvent per connection-state transition; error_handler receives one SenderError per server rejection (without it rejections are logged through the questdb logger). Each runs on its own dispatcher thread; see QuestDB.from_conf() for details. When a handler or listener closes over the returned handle, call QuestDB.close() explicitly (or use with) rather than leaving the handle to the garbage collector.

import questdb

with questdb.connect('ws::addr=localhost:9000;') as db:
    with db.sender() as sender:
        sender.row('trades', symbols={'sym': 'BTC-USD'},
                   columns={'price': 62000.0}, at=ts)
    db.dataframe(df, table_name='trades', at='ts')
    result = db.query('SELECT * FROM trades LIMIT 10')

with questdb.connect(host='localhost', port=9000) as db:
    ...

class questdb.QuestDB¶

Bases: object

Handle to a QuestDB deployment over QWP/WebSocket.

Owns the connection pool; lends row-building senders via sender(), bulk-loads DataFrames via dataframe(), runs queries via query(), and lends reader leases via reader(). Construct with questdb.connect(). Instances are safe to share across threads.

__enter__()¶

__exit__(exc_type, _exc_val, _exc_tb)¶

close()¶

Close the client and its connection pool.

This method is idempotent. When called from inside one of this handle’s own error_handler / connection_listener callbacks, it does not wait for a concurrent close() on another thread to finish; the in-flight callback completes after that close returns.

connection_events_delivered¶: Total connection events delivered to the listener. 0 when no listener is registered.

connection_events_dropped¶: Total connection events discarded by the listener inbox’s drop-oldest policy. 0 when no listener is registered.

Ingest a dataframe through the pooled columnar QWP path.

Ingestion always uses the direct (non-store-and-forward) column sender, independent of sf_dir, and returns once the whole frame is committed (AckLevel::Ok). On a transient connection failure the frame is re-sent from the caller’s DataFrame within the pool’s reconnect budget — unless the native client reports delivery as in doubt, or an intermediate commit checkpoint (emitted every ~100 batches on large frames) has already landed. In either case the error is raised immediately and a committed prefix may remain in the table, since a blind re-send would duplicate it. Delivery is at-least-once: a re-sent frame can duplicate already-committed rows (including the first chunk, which commits eagerly on a fresh connection) unless the table has DEDUP UPSERT KEYS. Server-side rejections (e.g. a schema mismatch) surface as a plain QuestDBError; the structured sender_error diagnostic is attached only by the store-and-forward senders.

df accepts any of:

pandas pandas.DataFrame. NumPy-backed columns route through the legacy planner; pyarrow-backed columns route through the Arrow C Stream capsule path below.
polars polars.DataFrame and polars.LazyFrame. LazyFrame is materialised via .collect(engine='streaming') (eager .collect() on polars < 1.0).
pyarrow pa.Table, pa.RecordBatch, and pa.RecordBatchReader.
Any object exposing the Arrow C Data Interface — i.e. with __arrow_c_stream__ (duckdb / cudf / modin / pyarrow-backed pandas 2.2+) or __arrow_c_array__ (single Arrow array exporters, wrapped into a one-batch pa.Table).

at names the designated timestamp column (by name or index). Alternatively pass a fixed TimestampNanos / datetime shared by every row (encoded as a repeated constant; resubmission stays idempotent under DEDUP UPSERT KEYS), or the explicit ServerTimestamp sentinel to let the server assign each row’s timestamp on arrival — an opt-in mirroring the row API, since server-assigned timestamps defeat DEDUP UPSERT KEYS on resubmission.

The columnar path loads one table per call: name it via table_name — or, for NumPy-backed pandas input, the dataframe’s index name (df.index.name); Arrow-native input (polars, pyarrow, pyarrow-backed pandas) requires an explicit table_name. table_name_col raises UnsupportedDataFrameShapeError — split multi-table frames (e.g. df.groupby(col)) and load each group.

Supports a column-QWP v1 subset: a single per-call table name, non-null designated timestamp column, and the following per-column dtypes:

Numeric: NumPy bool/int{8,16,32,64}/uint{8..64}/float{32,64}. Arrow pa.int{8,16,32,64}, pa.float{16,32,64}, and pa.uint{8,16,32,64} are accepted by the Rust Arrow batch route when the frame uses a fixed table name and a designated timestamp column name. Unsigned Arrow values follow the Rust Arrow policy: UInt8 and UInt16 widen to INT, UInt32 to LONG, and UInt64 values up to i64::MAX are accepted as LONG. Larger UInt64 values are rejected because QuestDB QWP-WS encodes integers as signed i64. Signed int8/int16 land as QuestDB INT; the row-oriented Sender.dataframe() instead widens every integer to LONG. Likewise float32 lands here as FLOAT but is widened to DOUBLE by the row path — ingest a given table through a single path to avoid a first-write column-type mismatch.
String / Symbol: object-dtype str, pa.string(), pa.large_string(), pd.CategoricalDtype of strings.
Timestamp: NumPy datetime64 units accepted by pandas and pa.timestamp with unit s, ms, us, or ns (tz-aware accepted on Arrow-backed columns in the Rust Arrow route). Timestamp field columns accept null timestamps (NaT / None) and values before the Unix epoch; a datetime64[ns] field carrying NaT is re-exported through Arrow so its INT64_MIN null sentinel is preserved (this needs pyarrow). The designated timestamp column (at=) must be non-null and at or after the epoch.
Decimal: Arrow-backed pa.decimal{32,64,128,256} columns (pa.decimal32/pa.decimal64 require pyarrow >= 18), or object-dtype columns of decimal.Decimal (re-exported through Arrow; the decimal width and scale are inferred from the values).
Array: Arrow pa.list_ / pa.large_list / pa.fixed_size_list columns with a float64 leaf (nested for multi-dimensional), and object-dtype columns of float64 numpy.ndarray cells (any rank; requires pyarrow). Both land as QuestDB ARRAY(DOUBLE). Null rows are allowed; null elements inside an array are not.
UUID: pa.fixed_size_binary(16) and the arrow.uuid extension type. Bytes are forwarded verbatim as QuestDB’s UUID wire layout (“bytes 0..8 lo half LE, bytes 8..16 hi half LE”), matching the convention shared across the c-questdb-client family (Rust direct, Polars). Round-trip is byte-identity at this layout; users who want uuid.UUID.bytes (RFC 4122 big-endian) round-trip must convert at their boundary.

Server-side coercion handles cross-type writes (e.g. pa.string() UUIDs landing in a UUID column are parsed server-side; narrow ints landing in a wider column are widened). Failures surface as QuestDBError from the flush().

schema_overrides reclassifies columns by name, mapping each to 'symbol', 'ipv4', 'char', or 'geohash' (e.g. {'venue': 'symbol', 'src_ip': 'ipv4'}). Unknown column names are rejected. It requires the Arrow columnar path (fully Arrow-backed input without table_name_col); on input that falls back to the NumPy planner it raises UnsupportedDataFrameShapeError. max_rows_per_batch sets the pipelining granularity, not a safety limit: any batch exceeding the negotiated per-batch byte cap is split regardless of it, and a single row is never bounded by it. Each batch is one unit of client memory and server-side apply, and a commit checkpoint fires every ~100 batches, so max_rows_per_batch * 100 rows is the replay window on a transient failover. Raise it for narrow numeric rows; lower it for very wide rows or tight memory. Streaming Arrow input (pa.RecordBatchReader) is not re-batched — the producer’s batch size governs.

error_events_delivered¶: Total server rejections delivered to the error_handler (or to the default logging handler when none was registered).

error_events_dropped¶: Total server rejections discarded by the handler inbox’s drop-oldest policy.

execute(sql, binds=None)¶

Run a statement and discard whatever it returns.

Executes sql like query(), drains the result to its clean end and returns the pooled connection — the pattern DDL and DML otherwise need spelled out as with db.query(sql) as r: r.to_pandas(). Statement output (a COPY status row, admin-function rows, a stray SELECT) is discarded; use query() when you want the result. The connection’s SYMBOL dictionary is left untouched.

binds behaves exactly as on query(). Returns None: the protocol carries no rows-affected count.

static from_conf(conf_str, *, connection_listener=None, connection_event_inbox_capacity=0, error_handler=None, error_event_inbox_capacity=0)¶

Construct a handle from a QWP/WebSocket configuration string.

Prefer the questdb.connect() module-level factory.

By default the handle connects eagerly: construction pre-opens the warm minimums (sender_pool_min sender and query_pool_min reader connections, one of each by default), so an unreachable server or bad credentials fail here, fast. With lazy_connect=true construction opens no connection: sender leases buffer locally and connect in the background, readers connect on first use (query_pool_min defaults to 0), and errors surface from the first operation instead. Combining lazy_connect=true with a blocking initial_connect_retry or a positive query_pool_min is a configuration conflict.

Pooled row auto-flush is enabled by default at 1,000 rows, 100 milliseconds, or an estimated encoded size at 90% of the effective frame cap. Before a server cap is known, or when the server omits it, the byte threshold is the lower of 8 MiB and 90% of the local store-and-forward frame cap. Set auto_flush_bytes=off to disable only the byte trigger, or auto_flush=off to disable all automatic publishing. The mode is immutable and shared by every lease; the interval starts when the first row enters an empty lease buffer. Auto-triggered publishes do not wait for server acknowledgement.

The underlying connection pool is opened by questdb_db_connect_with_handlers. Dataframe ingestion always uses the direct (non-store-and-forward) QWP/WebSocket column sender, independent of sf_dir. On a transient connection failure the frame is re-sent from the caller’s DataFrame only when the failed operation is provably not delivered. A delivery-unknown failure surfaces as QuestDBError with in_doubt set, because blindly re-sending could duplicate rows. request_timeout bounds each commit’s no-progress ack wait; request_timeout=0 disables that deadline, so a stalled but connected server can block dataframe() indefinitely.

connection_listener, when set, is a callable receiving one ConnectionEvent per connection-state transition of the pool’s ingress connections (initial connect, per-endpoint attempt failures, failover, terminal auth rejection). It runs on a dedicated dispatcher thread fed by a bounded inbox (connection_event_inbox_capacity; 0 selects the default of 64) with a drop-oldest overflow policy, so a slow listener cannot stall ingest or reconnects. Exceptions it raises are logged and swallowed. Dropped/delivered totals are available via connection_events_dropped / connection_events_delivered. Successful events are queued only after negotiated state, including the server frame cap, is committed; they do not acknowledge data.

error_handler, when set, is a callable receiving one SenderError per server rejection recorded by any of the pool’s store-and-forward connections — including rejections for rows published through a PooledSender that was already closed. It runs on its own dedicated dispatcher thread fed by a bounded inbox (error_event_inbox_capacity; 0 selects the default of 64, overflow drops the oldest event). Exceptions it raises are logged and swallowed. Without a handler every rejection is logged through the questdb logger instead — ERROR for terminal rejections, WARNING for retriable ones (the affected rows are replayed, not lost) — so rejections are never silent. A terminal rejection is queued only after the connection’s terminal latch and pollable diagnostic have been committed. Delivered/dropped totals are available via error_events_delivered / error_events_dropped. Use the handler for dead-lettering, alerting, and metrics; terminal failures also surface as QuestDBError from the sender calls themselves.

When a handler or listener closes over the returned handle, call close() explicitly (or use with): until closed, such a handle is pinned alive rather than collected, so an abandoned one leaks instead of crashing. The pin is dropped at interpreter shutdown, so an unclosed self-referential handle can still crash during finalization.

query(sql, binds=None, *, reset_symbol_dict=True)¶

Execute a SQL query and return a QueryResult.

Egress goes through the QuestDB Wire Protocol (QWP/WebSocket) /read/v1 endpoint. The reader is borrowed from the same connection pool that hosts the ingress writers and is returned to the pool when the returned QueryResult is consumed or closed (a poisoned connection is dropped instead). Auth / TLS settings apply to both directions.

To run several queries on one pooled connection, lease a PooledReader with reader() instead.

Parameters:

sql – SQL text to execute. Forwarded verbatim to QuestDB.
binds –
Positional bind parameters matching the $1..``$N`` placeholders in sql, as a list or tuple. Always prefer binds over interpolating values into the SQL text — they take no escaping and keep types exact. Supported Python types and their QuestDB bind types:
- None → SQL NULL (bound as a VARCHAR null)
- bool → BOOLEAN
- int → LONG (must fit signed 64-bit)
- float → DOUBLE
- str → VARCHAR
- datetime.datetime → TIMESTAMP (microseconds; a naive value is interpreted as UTC, the same rule as everywhere else in the API)
- TimestampMicros → TIMESTAMP
- TimestampNanos → TIMESTAMP_NS
- uuid.UUID → UUID
Any other type raises TypeError naming the placeholder.
```
res = db.query(
    'SELECT * FROM trades WHERE ts > $1 AND sym = $2',
    [datetime.datetime(2026, 7, 1), 'BTC-USD'])
```
reset_symbol_dict – When True (the default), the server resets the connection’s SYMBOL dictionary before this query so it never inherits symbols from earlier queries on the pooled connection (query-scoped dict), avoiding cross-query dictionary bloat in to_polars() / to_pandas(). Set False to keep the dictionary warm across repeated identical queries. No-op against servers that predate the capability.

Returns:

A QueryResult. Materialise it via to_pandas(), to_arrow(), iter_arrow(), iter_pandas(), or the __arrow_c_stream__ PyCapsule protocol.

Sentinel-value collisions in the result frame round-trip QuestDB’s contract: INT64_MIN in a LONG column, NaN in DOUBLE / FLOAT, and the sentinel values for INT / DATE / TIMESTAMP / TIMESTAMP_NS / CHAR / UUID / LONG256 / IPV4 / GEOHASH are all interpreted as NULL by QuestDB and cannot be distinguished from legitimate occurrences of those values.

reader()¶

Borrow a context-managed PooledReader lease from the pool.

The read-side twin of sender(): the lease holds one pooled reader connection for its lifetime and runs queries on it sequentially via PooledReader.query().

with db.reader() as r:
    r1 = r.query('SELECT * FROM t1').to_pandas()
    r2 = r.query(
        'SELECT * FROM t2',
        reset_symbol_dict=False).to_pandas()

The lease participates in the handle’s active-use count until it is closed. QuestDB.close() therefore waits for outstanding leases.

reap_idle()¶: Manually reap idle above-pool-size connections.

sender()¶

Borrow a context-managed row-building sender from the pool.

The lease participates in the handle’s active-use count until it is closed. QuestDB.close() therefore waits for outstanding leases.

server_info() → ServerInfo¶

Return a ServerInfo snapshot of the server’s SERVER_INFO handshake: cluster role, failover epoch, negotiated capabilities, handshake wall-clock, and cluster/node identifiers.

A reader is borrowed from the connection pool (opening one on first use, exactly like query()), sampled, and returned to the pool. The snapshot describes that connection at its last handshake; a later failover is reflected only in snapshots taken after it.

class questdb.PooledSender¶

Bases: object

A row-building sender borrowed from a QuestDB pool.

Obtain a lease with QuestDB.sender(); close() returns the native sender to the pool. Rows publish into an ordered, store-and-forward-covered QWP stream over one pooled connection.

Rows go through row(), dataframe(), flush(), wait() and close(); len(sender) is the number of buffered rows. Frame-level delivery tracking is available through flush_and_get_fsn(), flush_and_keep_and_get_fsn(), published_fsn(), acked_fsn() and await_acked_fsn(), and per-lease server diagnostics through poll_error() and error_events_dropped(). Prefer QuestDB.dataframe() for bulk loads; the lease’s own dataframe() is a convenience that routes to the same direct columnar path. Row auto-flush is enabled by default at 1,000 rows, 100 milliseconds, or a cap-derived byte threshold, and can be configured through the parent handle’s connection settings.

__enter__()¶

__exit__(exc_type, _exc_val, _exc_tb)¶

acked_fsn()¶: Highest FSN completed on the lease’s pooled connection by ACK or drop-and-continue rejection, or None if no frame has completed yet. A completed frame is not necessarily applied: a rejected frame the queue drops to make progress also advances this watermark.

await_acked_fsn(fsn, timeout_millis=0)¶

Wait until the acknowledgement watermark reaches fsn (as returned by flush_and_get_fsn() on this lease).

Returns True once fsn is acknowledged, or False if the no-progress timeout elapsed before fsn was acknowledged (0 waits indefinitely). Only a terminal connection failure raises.

close(flush: bool = True, wait: bool = False)¶

Return this sender to its pool. Idempotent.

Pending rows are published by default; delivery is owned by the store-and-forward queue, which keeps delivering after the sender is returned. wait=True additionally waits for an OK ack before returning the sender to the pool; without it, a later server rejection of this lease’s rows is reported through the pool’s error_handler (default: the questdb logger).

Bulk-load a whole DataFrame over a direct columnar connection borrowed from the pool for the duration of this call.

Prefer QuestDB.dataframe() — this convenience forwards to the same path and is not part of this sender’s row stream: the frame is committed over its own connection and becomes visible to SQL immediately, without waiting for rows appended to this sender with row() to drain. There is therefore no ordering relationship between dataframe() and buffered rows — dataframe() does not flush them; publish those with flush().

Arguments mirror QuestDB.dataframe().

error_events_dropped()¶: Diagnostics dropped from the lease’s connection ring (error_inbox_capacity).

flush(*, wait=False)¶

Publish and clear buffered rows.

By default this returns after local store-and-forward acceptance. Pass wait=True to wait for the server’s OK acknowledgement of everything published through this lease. The wait is a pure ack barrier: only a terminal connection failure raises. Server rejections are delivered to the pool’s error_handler (default: the questdb logger) instead; retriable ones are replayed by the store-and-forward queue.

flush_and_get_fsn()¶

Publish and clear buffered rows, returning the frame sequence number (FSN) of the published frame, or None if the buffer was empty.

FSNs are watermarks of the lease’s pooled connection: use them for progress tracking while this lease is held (see await_acked_fsn()); they are not portable receipts across leases, which may borrow different connections. Configure auto_flush=off when this call must publish and identify one whole application batch; auto-flush may already have published and cleared some or all buffered rows.

flush_and_keep_and_get_fsn()¶: Publish buffered rows without clearing the buffer, returning the published frame’s FSN, or None if the buffer was empty.

poll_error()¶

Poll the next server-rejection diagnostic recorded on the lease’s connection since this lease was borrowed, as a SenderError, or None when none is pending.

The pool’s error_handler independently receives every rejection the moment it is recorded; polling here is a per-lease pull alternative for code that wants diagnostics inline.

published_fsn()¶: Highest FSN published locally on the lease’s pooled connection, or None if nothing has been published on it yet.

Append one row to this sender’s QWP buffer.

When pooled auto-flush is enabled on the QuestDB handle, completing a row that breaches a configured row, byte, or interval threshold publishes the buffer without waiting for an acknowledgement. Any error raised by that publish propagates from this method. If a single row cannot fit in a QWP frame, that row is removed before QuestDBErrorCode.BatchTooLarge is raised. If a multi-row batch exceeds the exact encoded limit despite the byte-size estimate, the complete batch remains buffered.

wait(timeout_millis=0)¶

Wait for everything published through this lease to receive an OK ack; returns immediately if this lease published nothing.

The wait is a pure ack barrier: only a terminal connection failure raises. Server rejections are delivered to the pool’s error_handler (default: the questdb logger) instead.

timeout_millis is a no-progress timeout; 0 waits indefinitely.

class questdb.PooledReader¶

Bases: object

A reader lease borrowed from a QuestDB pool.

The read-side twin of QuestDB.sender(): obtain a lease with QuestDB.reader(); it holds one pooled reader connection for its lifetime and runs queries on it sequentially via query(). close() (or leaving the with block) releases the connection: back to the pool if the last query was drained cleanly, dropped otherwise.

Queries are strictly sequential — one result at a time. Fully drain (or close()) each QueryResult before calling query() again; running the next query while the previous result is still open raises QuestDBError. Closing an undrained result terminates the lease; call result.cancel() before result.close() to preserve it.

Because every query shares one connection, passing reset_symbol_dict=False to follow-up queries keeps the connection’s SYMBOL dictionary warm across them instead of resetting it per query.

The lease counts as an active use of the QuestDB handle (QuestDB.close() waits for it) and has thread affinity: use one lease per thread, on the thread that created it.

__enter__()¶

__exit__(exc_type, _exc_val, _exc_tb)¶

close()¶

Release the lease’s reader connection. Idempotent.

A still-open (undrained) last result is freed first, which tears down the connection; a cleanly-drained connection is returned to the pool, any other is dropped and the pool refills on demand.

execute(sql, binds=None)¶

Run a statement on the lease’s connection and discard whatever it returns.

Mirrors QuestDB.execute(). The result is drained to its clean end, so the lease stays usable for the next call, and the connection’s SYMBOL dictionary is left untouched — an interleaved statement does not invalidate a warm dictionary built with reset_symbol_dict=False.

query(sql, binds=None, *, reset_symbol_dict=True) → QueryResult¶

Execute a SQL query on the lease’s connection and return a QueryResult.

sql, binds and reset_symbol_dict behave exactly as on QuestDB.query(), except the query runs on the reader this lease holds instead of a per-call pool borrow. The previous query’s result must be fully drained (or closed) first; reset_symbol_dict=False reuses the connection’s SYMBOL dictionary built up by the lease’s earlier queries.

class questdb.Sender¶

Bases: object

Ingest data into QuestDB over a single connection.

This is the connection-level API: one sender drives exactly one connection (ILP/HTTP, ILP/TCP, QWP/UDP, or a single QWP/WebSocket connection) and carries the point-to-point capabilities the deployment-level handle does not: HTTP transactions, UDP datagrams, and manual ws progress and buffer control. For pooled ingestion and queries, prefer questdb.connect().

See the Sending Data documentation for more information.

__enter__() → Sender¶: Call Sender.establish() at the start of a with block.

__exit__(exc_type, _exc_val, _exc_tb)¶

Flush pending and disconnect at the end of a with block.

If the with block raises an exception, any pending data will NOT be flushed.

This is implemented by calling Sender.close().

__init__(*args, **kwargs)¶

acked_fsn()¶: Highest QWP/WebSocket frame sequence number completed by ACK or drop-and-continue rejection.

auto_flush¶

Auto-flushing is enabled.

Consult the .auto_flush_rows, .auto_flush_bytes and .auto_flush_interval properties for the current active thresholds.

auto_flush_bytes¶: Byte-count threshold for the auto-flush logic, or None if disabled.

auto_flush_interval¶: Time interval threshold for the auto-flush logic, or None if disabled.

auto_flush_rows¶: Row count threshold for the auto-flush logic, or None if disabled.

await_acked_fsn(fsn, timeout_millis=0)¶

Wait until the QWP/WebSocket completion watermark reaches fsn.

Returns True once every frame published so far (which includes fsn) has been acknowledged, or False if the no-progress timeout elapsed before the acknowledgement watermark reached fsn.

close(flush=True)¶

Disconnect.

This method is idempotent and can be called repeatedly.

Once a sender is closed, it can’t be re-used.

Parameters:: flush (bool) – If True, flush the internal buffer before closing. For QWP/WebSocket, this also drains already-published frames before closing.

close_drain()¶: Stop accepting new QWP/WebSocket publications and wait for already published frames to resolve.

connection_events_delivered¶: Total connection events delivered to the listener. 0 when no listener is registered.

connection_events_dropped¶: Total connection events discarded by the listener inbox’s drop-oldest policy. 0 when no listener is registered.

Write a Pandas DataFrame to QuestDB.

Over ILP/HTTP and ILP/TCP the frame is serialized into the internal row buffer; over QWP/UDP into fire-and-forget datagrams, with the same delivery caveats as row(). Over QWP/WebSocket the frame is bulk-loaded through a poolless direct columnar connection opened from this sender’s configuration for the call (the same direct path as QuestDB.dataframe(), carrying the sender’s auth/TLS regardless of how it was constructed); max_rows_per_batch applies only on that path, and passing schema_overrides on any other protocol raises. The direct load has no ordering relationship with rows buffered via row() and does not flush them.

Example:

import pandas as pd
import questdb as qi

df = pd.DataFrame({
    'car': pd.Categorical(['Nic 42', 'Eddi', 'Nic 42', 'Eddi']),
    'position': [1, 2, 1, 2],
    'speed': [89.3, 98.2, 3, 4],
    'lat_gforce': [0.1, -0.2, -0.6, 0.4],
    'accelleration': [0.1, -0.2, 0.6, 4.4],
    'tyre_pressure': [2.6, 2.5, 2.6, 2.5],
    'ts': [
        pd.Timestamp('2022-08-09 13:56:00'),
        pd.Timestamp('2022-08-09 13:56:01'),
        pd.Timestamp('2022-08-09 13:56:02'),
        pd.Timestamp('2022-08-09 13:56:03')]})

with qi.Sender.from_env() as sender:
    sender.dataframe(df, table_name='race_metrics', at='ts')

See the buffer-level dataframe documentation for details on the supported column types and arguments.

Additionally, this method also supports auto-flushing the buffer as specified in the Sender’s auto_flush constructor argument. Auto-flushing is implemented incrementally, meanting that when calling sender.dataframe(df) with a large df, the sender may have sent some of the rows to the server already whist the rest of the rows are going to be sent at the next auto-flush or next explicit call to Sender.flush().

In case of data errors with auto-flushing enabled, some of the rows may have been transmitted to the server already.

drive_once()¶: Drive one QWP/WebSocket progress step for manual progress senders.

error_events_dropped()¶: Number of QWP/WebSocket diagnostics dropped from the bounded ring.

establish()¶

Prepare the sender for use.

If using ILP/HTTP this will initialize the HTTP connection pool.

If using ILP/TCP this will cause connection to the server and block until the connection is established.

If the TCP connection is set up with authentication and/or TLS, this method will return only after the handshake(s) is/are complete.

flush(buffer=None, clear=True, transactional=False)¶

If called with no arguments, immediately flushes the internal buffer.

Alternatively you can flush a buffer that was constructed explicitly by passing buffer.

The buffer will be cleared by default, unless clear is set to False.

This method does nothing if the provided or internal buffer is empty.

Parameters:: buffer – The buffer to flush. If None, the internal buffer is flushed.

With QWP/WebSocket, this publishes the buffer into the local sender queue and returns before the server necessarily ACKs the frame. Later terminal diagnostics fail subsequent sender calls and are available as QuestDBError.sender_error. Server diagnostics are also available through Sender.poll_error().

Parameters:

clear – If True, the flushed buffer is cleared (default). If False, the flushed buffer is left in the internal buffer. Note that clear=False is only supported if buffer is also specified.
transactional – If True ensures that the flushed buffer contains row for a single table, ensuring all data can be written transactionally. This feature requires ILP/HTTP and is not available when connecting over TCP. Default: False.

The Python GIL is released during the network IO operation.

flush_and_get_fsn(buffer=None)¶: Publish a QWP/WebSocket buffer locally, clear it on success, and return the assigned frame sequence number.

flush_and_keep_and_get_fsn(buffer=None)¶: Publish a QWP/WebSocket buffer locally without clearing it and return the assigned frame sequence number.

static from_conf(conf_str, *, bind_interface=None, username=None, password=None, token=None, token_x=None, token_y=None, auth_timeout=None, tls_verify=None, tls_ca=None, tls_roots=None, tls_roots_password=None, max_buf_size=None, retry_timeout=None, retry_max_backoff=None, request_min_throughput=None, request_timeout=None, auto_flush=None, auto_flush_rows=None, auto_flush_bytes=None, auto_flush_interval=None, max_datagram_size=None, multicast_ttl=None, qwp_ws_progress=None, error_handler=None, connection_listener=None, connection_event_inbox_capacity=0, protocol_version=None, init_buf_size=None, max_name_len=None)¶

Construct a sender from a configuration string.

The additional arguments are used to specify additional parameters which are not present in the configuration string.

Note that any parameters already present in the configuration string cannot be overridden.

static from_env(*, bind_interface=None, username=None, password=None, token=None, token_x=None, token_y=None, auth_timeout=None, tls_verify=None, tls_ca=None, tls_roots=None, tls_roots_password=None, max_buf_size=None, retry_timeout=None, retry_max_backoff=None, request_min_throughput=None, request_timeout=None, auto_flush=None, auto_flush_rows=None, auto_flush_bytes=None, auto_flush_interval=None, max_datagram_size=None, multicast_ttl=None, qwp_ws_progress=None, error_handler=None, connection_listener=None, connection_event_inbox_capacity=0, protocol_version=None, init_buf_size=None, max_name_len=None)¶

Construct a sender from the QDB_CLIENT_CONF environment variable.

The environment variable must be set to a valid configuration string.

The additional arguments are used to specify additional parameters which are not present in the configuration string.

Note that any parameters already present in the configuration string cannot be overridden.

init_buf_size¶: The initial capacity of the sender’s internal buffer.

max_name_len¶: Maximum length of a table or column name.

new_buffer()¶

Make a new configured buffer.

The buffer is set up with the configured init_buf_size and max_name_len, and matches the sender’s protocol.

Must be called after Sender.establish() and before Sender.close(); otherwise raises QuestDBError (InvalidApiCall).

poll_error()¶: Poll the next structured QWP/WebSocket diagnostic.

protocol_version¶

The protocol version used by the sender.

Protocol version 1 is retained for backwards compatibility with older QuestDB versions.

Protocol version 2 introduces binary floating point support and the array datatype.

published_fsn()¶: Highest QWP/WebSocket frame sequence number published locally.

Write a row to the internal buffer.

This may be sent automatically depending on the auto_flush setting in the constructor.

Refer to the Buffer.row documentation for details on arguments.

Note: Support for NumPy arrays (numpy.array) requires QuestDB server version 9.0.0 or higher.

transaction(table_name: str)¶: Start a HTTP Transactions block.

class questdb.SenderTransaction¶

Bases: object

A transaction for a specific table.

Transactions are only supported with ILP/HTTP.

The sender API can only operate on one transaction at a time.

To create a transaction:

with sender.transaction('table_name') as txn:
    txn.row(..)
    txn.dataframe(..)

__enter__()¶

__exit__(exc_type, _exc_value, _traceback)¶

commit()¶

Commit the transaction.

A commit is also automatic at the end of a successful with block.

This will flush the buffer.

Write a dataframe for the table in the transaction.

The table name is taken from the transaction.

rollback()¶

Roll back the transaction.

A rollback is also automatic at the end of a failed with block.

This will clear the buffer.

Write a row for the table in the transaction.

The table name is taken from the transaction.

Note: Support for NumPy arrays (numpy.array) requires QuestDB server version 9.0.0 or higher.

class questdb.QueryResult(cursor_handle)¶

Bases: object

Result of QuestDB.query(sql).

Streams query rows as Arrow record batches. Single-use: each materialisation method (to_pandas, to_arrow, iter_arrow, iter_pandas, or the __arrow_c_stream__ PyCapsule protocol) consumes the underlying cursor; the second consumption raises QuestDBError.

Consumption rule: fully drain the result, use it as a context manager (with db.query(...) as result:), or call close(). Closing a partial result drops its connection. Call cancel() before closing to preserve it.

Thread affinity: the underlying cursor is bound to the thread that created it (via QuestDB.query). Create, consume, cancel, close, and drop the QueryResult on that same thread; handing it to another thread is undefined behaviour even with external synchronisation.

__arrow_c_stream__ is native — the cursor’s record batches are exposed directly through the Arrow C Data Interface, so polars / duckdb / pandas 3.0 / any Arrow-native consumer can read query results without pyarrow installed. to_pandas / iter_pandas are also pyarrow-free by default (a native numpy path); they only need pyarrow when dtype_backend / types_mapper is passed. to_arrow / iter_arrow / to_polars / iter_polars do require pyarrow.

SYMBOL columns: to_polars / to_pandas build the Categorical directly, interning the connection dictionary once (no per-row remap). to_arrow / iter_arrow / __arrow_c_stream__ emit a generic Arrow form whose per-batch SYMBOL dictionary is compacted to the values each batch uses, which a generic consumer reconciles. So when the target is a polars / pandas frame, the dedicated methods avoid the re-reconciliation that polars.from_arrow(result) / to_arrow().to_pandas() pay on SYMBOL-heavy results.

Example:

with db.query('SELECT * FROM trades WHERE ts > $1',
              [datetime.datetime(2026, 7, 1)]) as result:
    df = polars.from_arrow(result)              # no pyarrow
    # df = result.to_pandas()                   # no pyarrow
    # table = pa.table(result)                  # pyarrow required

__enter__()¶

__exit__(exc_type, exc_val, exc_tb)¶

__init__(cursor_handle)¶

cancel()¶

Cancel the query and drain to terminal.

The result remains open; call close() before starting another query on the same reader. On failure, the result is closed. Idempotent after success.

close()¶

Release the cursor and reader. Idempotent.

Terminal connections return to the pool; others are dropped. This does not cancel the query. Outstanding iterators become invalid.

iter_arrow()¶

Iterate result batches as pyarrow.RecordBatch.

Streaming: a mid-query failover after the first batch has been yielded surfaces QuestDBErrorCode.FailoverWouldDuplicate (the already-yielded batches cannot be discarded); re-issue the query. If the iterator is abandoned partway, cleanup runs at the next garbage-collection cycle; call close() (or use the context- manager) for deterministic release.

iter_pandas(*, dtype_backend=None, types_mapper=None)¶

Iterate result batches as pandas.DataFrame.

Mirrors to_pandas(): with no arguments each batch is materialised straight into numpy (no pyarrow, sentinel-preserving, df.attrs['questdb'] per batch). dtype_backend / types_mapper select the pyarrow-backed path instead.

iter_polars()¶

Iterate result batches as polars.DataFrame.

Mirrors to_polars() per batch (same Categorical SYMBOL handling) for streaming / low-peak-memory consumption. Every batch’s SYMBOL Categoricals share one persistent Categories identity, so polars.concat over the yielded frames stitches without a categories-mismatch error.

Streaming: a mid-query failover after the first batch has been yielded surfaces QuestDBErrorCode.FailoverWouldDuplicate; re-issue the query. Requires polars and pyarrow.

to_arrow()¶

Read the full result into a pyarrow.Table. Requires pyarrow.

Materialise-whole: a mid-query failover replays the result transparently — the partial accumulation we hold is discarded from batch-0. The pyarrow-free streaming path (__arrow_c_stream__ consumed by polars.from_arrow(result) / pa.table(result)) instead surfaces FailoverWouldDuplicate on a post-delivery failover.

to_pandas(*, dtype_backend=None, types_mapper=None)¶

Read the full result into a pandas.DataFrame.

The default is a native (no pyarrow) hybrid built straight from the QWP column buffers: a nullable integer column with nulls becomes a pandas nullable Int* (pd.NA); without nulls it stays plain numpy. double/float stay numpy with NaN; SYMBOL → Categorical; TIMESTAMP → datetime64 (NaT); strings/decimal/uuid/binary → object. Analysis-safe (aggregations skip pd.NA/NaN), and feeds back into QuestDB.dataframe() for a type round-trip — the column kinds are carried in df.attrs['questdb'].

dtype_backend="pyarrow" / "numpy_nullable" / types_mapper select the pyarrow-backed path instead (pd.ArrowDtype, pandas nullable extension dtypes, or a custom mapper) — matching the pd.read_sql / pd.read_parquet convention.

to_polars()¶

Read the full result into a polars.DataFrame. Requires polars and pyarrow.

Non-SYMBOL columns keep their exact polars.from_arrow dtypes (tz-aware Datetime, Decimal, Binary, List/Array, …). SYMBOL columns are built into a polars Categorical directly from their codes + dictionary through a persistent Categories registry — the wire code is its own physical categorical code, so there is no per-row Dictionary -> Categorical remap. Falls back to polars.from_arrow when polars’ (unstable) Categories API is unavailable.

Materialise-whole: a mid-query failover replays the result transparently. This accumulates batches in-library (via pyarrow) so the partial result can be discarded on failover; for the pyarrow-free streaming path consume __arrow_c_stream__ directly (polars.from_arrow(result)), which surfaces FailoverWouldDuplicate on a post-delivery failover.

Bases: object

One connection-state transition, delivered to the connection_listener registered via QuestDB.from_conf().

Listeners run on a dedicated dispatcher thread — never on an I/O or caller thread — fed by a bounded inbox with a drop-oldest overflow policy, so a slow listener cannot stall ingest or reconnects. Successful events are queued only after negotiated connection state, including the server-advertised frame cap, is committed. They are not data-delivery or acknowledgement barriers. Exceptions raised by the listener are logged and swallowed.

__init__(kind: ConnectionEventKind, host: str | None, port: str | None, previous_host: str | None, previous_port: str | None, attempt_number: int | None, cause_code: QuestDBErrorCode | None, cause_msg: str | None, timestamp_millis: int) → None¶

attempt_number: int | None¶

cause_code: QuestDBErrorCode | None¶

cause_msg: str | None¶

host: str | None¶

kind: ConnectionEventKind¶

port: str | None¶

previous_host: str | None¶

previous_port: str | None¶

timestamp_millis: int¶

class questdb.ConnectionEventKind(*values)¶

Bases: TaggedEnum

Connection-state transitions observed by the ingress connection pool.

AllEndpointsUnreachable = ('all_endpoints_unreachable', 5)¶

AuthFailed = ('auth_failed', 6)¶

Connected = ('connected', 0)¶

Disconnected = ('disconnected', 1)¶

EndpointAttemptFailed = ('endpoint_attempt_failed', 4)¶

FailedOver = ('failed_over', 3)¶

Reconnected = ('reconnected', 2)¶

class questdb.ServerInfo(role: ServerRole, role_byte: int, epoch: int, capabilities: int, server_wall_ns: int, cluster_id: str, node_id: str, zone_id: str | None)¶

Bases: object

Snapshot of the server’s SERVER_INFO handshake, as advertised on the connection QuestDB.server_info() sampled. Fields mirror the Rust reader’s ServerInfo.

__init__(role: ServerRole, role_byte: int, epoch: int, capabilities: int, server_wall_ns: int, cluster_id: str, node_id: str, zone_id: str | None) → None¶

capabilities: int¶

cluster_id: str¶

epoch: int¶

node_id: str¶

role: ServerRole¶

role_byte: int¶

server_wall_ns: int¶

zone_id: str | None¶

class questdb.ServerRole(*values)¶

Bases: TaggedEnum

Cluster role advertised by the server’s SERVER_INFO handshake.

Other = ('other', 255)¶

Primary = ('primary', 1)¶

PrimaryCatchup = ('primary_catchup', 3)¶

Replica = ('replica', 2)¶

Standalone = ('standalone', 0)¶

class questdb.QuestDBError(code, msg, sender_error=None, *, in_doubt=False)¶

Bases: Exception

An error whilst using the QuestDB client.

__init__(code, msg, sender_error=None, *, in_doubt=False)¶

property code: QuestDBErrorCode¶: Return the error code.

property in_doubt: bool¶

Whether the failed operation may already have delivered its input.

Retrying the same input when this is true can duplicate rows unless the destination table has an appropriate deduplication guarantee.

property sender_error¶: Return the structured QWP/WebSocket diagnostic, if this error carries one from a QWP/WebSocket sender failure.

class questdb.QuestDBServerRejectionError(code, msg, sender_error=None, *, in_doubt=False)¶

Bases: QuestDBError

A terminal QWP/WebSocket server rejection.

The structured server payload is available through QuestDBError.sender_error.

class questdb.UnsupportedDataFrameShapeError(msg, column_failures=None)¶

Bases: QuestDBError

A DataFrame shape is not supported by the optimized columnar client path.

column_failures carries structured per-column rejection details where available; each is a dict with column (name or None for a whole-frame issue), target, source_code, and reason. The same per-column reasons are also folded into str(exc) so a bare print(exc) explains why the frame was rejected, and — when the designated timestamp is at fault — how to fix it on QuestDB.dataframe() (name a valid timestamp column, or pass ServerTimestamp).

__init__(msg, column_failures=None)¶

class questdb.QuestDBErrorCode(*values)¶

Bases: Enum

Category of Error.

ArrayError = 11¶

ArrowExport = 34¶

ArrowIngest = 16¶

ArrowUnsupportedColumnKind = 15¶

AuthError = 6¶

BadDataFrame = 65536¶

BatchTooLarge = 35¶

Cancelled = 30¶

ConfigError = 10¶

ConnectTimeout = 19¶

CouldNotResolveAddr = 0¶

DecimalError = 13¶

FailoverRetry = 17¶

FailoverWouldDuplicate = 31¶

HandshakeError = 20¶

HttpNotSupported = 8¶

InvalidApiCall = 1¶

InvalidBind = 23¶

InvalidName = 4¶

InvalidTimestamp = 5¶

InvalidUtf8 = 3¶

LimitExceeded = 28¶

NoSchema = 33¶

ProtocolError = 22¶

ProtocolVersionError = 12¶

RoleMismatch = 18¶

SchemaDrift = 32¶

ServerFlushError = 9¶

ServerInternalError = 26¶

ServerLimitExceeded = 29¶

ServerParseError = 25¶

ServerRejection = 14¶

ServerSchemaMismatch = 24¶

ServerSecurityError = 27¶

SocketError = 2¶

StoreResendRequired = 36¶

TlsError = 7¶

UnsupportedServer = 21¶

__str__() → str¶: Return the name of the enum.

class questdb.SenderError(category: SenderErrorCategory, applied_policy: SenderErrorPolicy, status: int | None, message: str, message_sequence: int | None, from_fsn: int, to_fsn: int)¶

Bases: object

Structured QWP/WebSocket server diagnostic.

__init__(category: SenderErrorCategory, applied_policy: SenderErrorPolicy, status: int | None, message: str, message_sequence: int | None, from_fsn: int, to_fsn: int) → None¶

applied_policy: SenderErrorPolicy¶

category: SenderErrorCategory¶

from_fsn: int¶

message: str¶

message_sequence: int | None¶

status: int | None¶

to_fsn: int¶

class questdb.SenderErrorCategory(*values)¶

Bases: TaggedEnum

Category of a structured QWP/WebSocket diagnostic.

InternalError = ('internal_error', 2)¶

NotWritable = ('not_writable', 5)¶

ParseError = ('parse_error', 1)¶

ProtocolViolation = ('protocol_violation', 6)¶

SchemaMismatch = ('schema_mismatch', 0)¶

SecurityError = ('security_error', 3)¶

Unknown = ('unknown', 7)¶

WriteError = ('write_error', 4)¶

class questdb.SenderErrorPolicy(*values)¶

Bases: TaggedEnum

Applied policy for a structured QWP/WebSocket diagnostic.

Retriable = ('retriable', 0)¶

RetriableOther = ('retriable_other', 1)¶

Terminal = ('terminal', 2)¶

class questdb.QwpWsProgress(*values)¶

Bases: TaggedEnum

Progress mode for QWP/WebSocket senders.

Background = ('background', 0)¶

Manual = ('manual', 1)¶

class questdb.Protocol(*values)¶

Bases: TaggedEnum

Protocol to use for sending data to QuestDB.

See Which protocol? for more information.

Http = ('http', 2)¶

Https = ('https', 3)¶

Tcp = ('tcp', 0)¶

Tcps = ('tcps', 1)¶

Udp = ('udp', 4)¶

Ws = ('ws', 5)¶

Wss = ('wss', 6)¶

property tls_enabled¶

class questdb.TimestampMicros¶

Bases: object

A timestamp in microseconds since the UNIX epoch (UTC).

You may construct a TimestampMicros from an integer or a datetime.datetime, or simply call the TimestampMicros.now() method.

# Recommended way to get the current timestamp.
TimestampMicros.now()

# The above is equivalent to:
TimestampMicros(time.time_ns() // 1000)

# You can provide a numeric timestamp too. It can't be negative.
TimestampMicros(1657888365426838)

TimestampMicros can also be constructed from a datetime.datetime object.

TimestampMicros.from_datetime(
    datetime.datetime.now(tz=datetime.timezone.utc))

We recommend that when using datetime objects, you explicitly pass in the timezone to use. A datetime object without an associated timezone is interpreted as UTC (a UserWarning is emitted once per process). Note that datetime.datetime.now() is your local wall clock: use datetime.datetime.now(datetime.timezone.utc) or now() on this class for the current instant.

classmethod from_datetime(dt: datetime.datetime)¶: Construct a TimestampMicros from a datetime.datetime object.

classmethod now()¶: Construct a TimestampMicros from the current time as UTC.

value¶: Number of microseconds (Unix epoch timestamp, UTC).

class questdb.TimestampNanos¶

Bases: object

A timestamp in nanoseconds since the UNIX epoch (UTC).

You may construct a TimestampNanos from an integer or a datetime.datetime, or simply call the TimestampNanos.now() method.

# Recommended way to get the current timestamp.
TimestampNanos.now()

# The above is equivalent to:
TimestampNanos(time.time_ns())

# You can provide a numeric timestamp too. It can't be negative.
TimestampNanos(1657888365426838016)

TimestampNanos can also be constructed from a datetime object.

TimestampNanos.from_datetime(
    datetime.datetime.now(tz=datetime.timezone.utc))

We recommend that when using datetime objects, you explicitly pass in the timezone to use. A datetime object without an associated timezone is interpreted as UTC (a UserWarning is emitted once per process). Note that datetime.datetime.now() is your local wall clock: use datetime.datetime.now(datetime.timezone.utc) or now() on this class for the current instant.

classmethod from_datetime(dt: datetime.datetime)¶: Construct a TimestampNanos from a datetime.datetime object.

classmethod now()¶: Construct a TimestampNanos from the current time as UTC.

value¶: Number of nanoseconds (Unix epoch timestamp, UTC).

class questdb.TlsCa(*values)¶

Bases: TaggedEnum

Verification mechanism for the server’s certificate.

Here webpki refers to the WebPKI library and os refers to the operating system’s certificate store.

See TLS for more information.

OsRoots = ('os_roots', 1)¶

PemFile = ('pem_file', 3)¶

WebpkiAndOsRoots = ('webpki_and_os_roots', 2)¶

WebpkiRoots = ('webpki_roots', 0)¶

class questdb.ServerTimestampType¶

Bases: object

A placeholder value to indicate that the data should be inserted using a server-generated-timestamp.

Don’t instantiate this class directly, use the singleton ServerTimestamp instead.

This feature is mostly provided for legacy compatibility. We recommend always specifying an explicit timestamp.

Using ServerTimestamp will prevent QuestDB’s deduplication feature from working as it would generate unique rows on resubmission.

questdb.ServerTimestamp¶

A placeholder value to indicate that the data should be inserted using a server-generated-timestamp.

Don’t instantiate this class directly, use the singleton ServerTimestamp instead.

This feature is mostly provided for legacy compatibility. We recommend always specifying an explicit timestamp.

Using ServerTimestamp will prevent QuestDB’s deduplication feature from working as it would generate unique rows on resubmission.

class questdb._client.TaggedEnum(new_class_name, /, names, *, module=None, qualname=None, type=None, start=1, boundary=None)¶

Bases: Enum

Base class for tagged enums.

property c_value¶

classmethod parse(tag)¶: Parse from the tag name.

property tag¶: Short name.

questdb.WARN_HIGH_RECONNECTS¶: Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

questdb.ingress (legacy)¶

The deprecated 4.x compatibility shim. New code imports from questdb.

class questdb.ingress.Buffer¶

Internal row-serialization buffer, managed by Sender.

Kept importable as questdb.ingress.Buffer for legacy ILP/HTTP and ILP/TCP code that constructs buffers explicitly and flushes them via sender.flush(buffer).

__init__(*args, **kwargs)¶

capacity() → int¶: The current buffer capacity.

clear()¶

Reset the buffer.

Note that flushing a buffer will (unless otherwise specified) also automatically clear it.

This method is designed to be called only in conjunction with sender.flush(buffer, clear=False).

Add a pandas DataFrame to the buffer.

Also see the Sender.dataframe method if you’re not using the buffer explicitly. It supports the same parameters and also supports auto-flushing.

Requires pandas and numpy. pyarrow is only needed when the frame contains pd.ArrowDtype / pd.Categorical / string dtype columns — purely NumPy / object dtypes work without it.

Adding a dataframe can trigger auto-flushing behaviour, even between rows of the same dataframe. To avoid this, you can use HTTP and transactions (see Sender.transaction).

Parameters:

df (pandas.DataFrame) – The pandas DataFrame to serialize to the buffer.
table_name (str or None) –
The name of the table to which the rows belong.

If None, the table name is taken from the table_name_col parameter. If both table_name and table_name_col are None, the table name is taken from the DataFrame’s index name (df.index.name attribute).
table_name_col (str or int or None) –
The name or index of the column in the DataFrame that contains the table name.

If None, the table name is taken from the table_name parameter. If both table_name and table_name_col are None, the table name is taken from the DataFrame’s index name (df.index.name attribute).

If table_name_col is an integer, it is interpreted as the index of the column starting from 0. The index of the column can be negative, in which case it is interpreted as an offset from the end of the DataFrame. E.g. -1 is the last column.
symbols (str or bool or list[str] or list[int]) –
The columns to be serialized as symbols.

If 'auto' (default), all columns of dtype 'categorical' are serialized as symbols. If True, all str columns are serialized as symbols. If False, no columns are serialized as symbols.

The list of symbols can also be specified explicitly as a list of column names (str) or indices (int). Integer indices start at 0 and can be negative, offset from the end of the DataFrame. E.g. -1 is the last column.

Only columns containing strings can be serialized as symbols.
at (TimestampNanos, datetime.datetime, int or str or None) –
The designated timestamp of the rows.

You can specify a single value for all rows or column name or index. If ServerTimestamp, timestamp is assigned by the server for all rows. To pass in a timestamp explicitly as an integer use the TimestampNanos wrapper type. To get the current timestamp, use TimestampNanos.now(). When passing a datetime.datetime object, the timestamp is converted to nanoseconds. A naive datetime object is interpreted as UTC — never your machine’s local timezone — and a UserWarning is emitted once per process (call datetime.datetime.now(tz=datetime.timezone.utc) for the current timestamp to avoid bugs).

To specify a different timestamp for each row, pass in a column name (str) or index (int, 0-based index, negative index supported): In this case, the column needs to be of dtype datetime64[ns] (assumed to be in the UTC timezone and not local, due to differences in Pandas and Python datetime handling) or datetime64[ns, tz]. When a timezone is specified in the column, it is converted to UTC automatically.

A timestamp column can also contain None values. The server will assign the current timestamp to those rows.

Note: All timestamps are always converted to nanoseconds and in the UTC timezone. Timezone information is dropped before sending and QuestDB will not store any timezone information.

Note: It is an error to specify both table_name and table_name_col.

Note: The “index” column of the DataFrame is never serialized, even if it is named.

Example:

import pandas as pd
import questdb as qi

buf = qi.ingress.Buffer(protocol_version=2)
# ...

df = pd.DataFrame({
    'location': ['London', 'Managua', 'London'],
    'temperature': [24.5, 35.0, 25.5],
    'humidity': [0.5, 0.6, 0.45],
    'ts': pd.date_range('2021-07-01', periods=3)})
buf.dataframe(
    df, table_name='weather', at='ts', symbols=['location'])

# ...
sender.flush(buf)

Pandas to ILP datatype mappings

Pandas `dtype`	Nulls	ILP Datatype
`'bool'`	N	`BOOLEAN`
`'boolean'`	N α	`BOOLEAN`
`'object'` (`bool` objects)	N α	`BOOLEAN`
`'uint8'`	N	`INTEGER`
`'int8'`	N	`INTEGER`
`'uint16'`	N	`INTEGER`
`'int16'`	N	`INTEGER`
`'uint32'`	N	`INTEGER`
`'int32'`	N	`INTEGER`
`'uint64'`	N	`INTEGER` β
`'int64'`	N	`INTEGER`
`'UInt8'`	Y	`INTEGER`
`'Int8'`	Y	`INTEGER`
`'UInt16'`	Y	`INTEGER`
`'Int16'`	Y	`INTEGER`
`'UInt32'`	Y	`INTEGER`
`'Int32'`	Y	`INTEGER`
`'UInt64'`	Y	`INTEGER` β
`'Int64'`	Y	`INTEGER`
`'object'` (`int` objects)	Y	`INTEGER` β
`'float32'` γ	Y (`NaN`)	`FLOAT`
`'float64'`	Y (`NaN`)	`FLOAT`
`'object'` (`float` objects)	Y (`NaN`)	`FLOAT`
`'string'` (`str` objects)	Y	`STRING` (default), `SYMBOL` via `symbols` arg. δ
`'string[pyarrow]'`	Y	`STRING` (default), `SYMBOL` via `symbols` arg. δ
`'category'` (`str` objects) ε	Y	`SYMBOL` (default), `STRING` via `symbols` arg. δ
`'object'` (`str` objects)	Y	`STRING` (default), `SYMBOL` via `symbols` arg. δ
`'datetime64[ns]'`	Y	`TIMESTAMP` ζ
`'datetime64[ns, tz]'`	Y	`TIMESTAMP` ζ
`'object'` (`Decimal` objects)	Y (`NaN`)	`DECIMAL`

API Reference¶

questdb¶

questdb.ingress (legacy)¶

questdb

Table of Contents

This Page

Python type	Serialized as ILP type
`bool`	BOOLEAN
`decimal`	DECIMAL
`int`	INTEGER
`float`	FLOAT
`str`	STRING
`numpy.ndarray`	ARRAY
`datetime.datetime` and `TimestampMicros`	TIMESTAMP
`None`	Column is skipped and not serialized.