agensgraph-python

Read and write an AgensGraph graph from Python, in blocking or awaiting code, over psycopg 3.

A vertex, an edge and a path come back as values you can use rather than strings you have to parse. Cypher and SQL run on the same connection, in the same transaction, so a graph traversal and a join are one query when you want them to be. Everything psycopg does (cursors, server-side cursors, COPY, pipelines, LISTEN, two-phase commit) keeps working, because the connection is a psycopg connection.

import agensgraph

with agensgraph.connect("host=localhost dbname=graph") as conn:
    conn.graph("social")
    result = conn.execute_query("MATCH (n:Person) RETURN n")
    for (person,) in result.records:
        print(person.properties["name"])

Install

pip install agensgraph-python

Python 3.11 or later. AgensGraph 2.17 or later.

Optional extras, imported only where they are used, so they never load on a read path:

pip install "agensgraph-python[arrow]"     # to_arrow
pip install "agensgraph-python[pandas]"    # to_pandas
pip install "agensgraph-python[polars]"    # to_polars
pip install "agensgraph-python[otel]"      # spans through opentelemetry-api

Connecting

import agensgraph

with agensgraph.connect("host=localhost dbname=graph") as conn:
    conn.graph("social")
    ...

The awaiting interface is the same interface, awaited:

conn = await agensgraph.AsyncConnection.connect("host=localhost dbname=graph")
async with conn:
    await conn.graph("social")
    result = await conn.execute_query("MATCH (n:Person) RETURN n")

Only the methods that wait for the server are written twice. Everything else (adapters, the label table, statement checks, building a result) exists once and is shared, and the blocking interface is generated from the awaiting one, so the two cannot drift.

AsyncConnection.connect is the awaiting entry point. The module-level agensgraph.connect() and agensgraph.dbapi.connect() are the blocking ones and have no awaiting form: connect() is Connection.connect under another name, and DBAPI is a blocking specification. Where a helper takes a connection and has to talk to it, the awaiting form is named for it: adapters.async_assert_oids(conn) beside adapters.assert_oids(conn).

conn.graph(name) selects a graph and fills the label table that the composite rendering needs. The name is quoted rather than bound, because the grammar has no place for a parameter there. The table is dropped again automatically when anything moves the session somewhere else, including a rollback that undoes the move.

Transactions are psycopg's, unchanged: a statement outside a transaction opens one, commit() and rollback() are on the connection, closing rolls back, and autocommit=True is how a statement that cannot run inside a transaction gets to run.

The server is checked at connect. The version arrives in the startup packet, so refusing a server this driver cannot read costs no round trip and happens before the first statement rather than at whichever later one first wanted a catalog the server has never had.

Running a statement

result = conn.execute_query("MATCH (n:Person) WHERE n.name = %s RETURN n", ("Arthur",))

result.records     # a list of tuples
result.keys        # the column names, read once per result
result.counts      # what the statement changed, if you asked
result.oids        # the type of each column, as the server described it

execute_query takes the statement, the parameters as a sequence, and reserved arguments ending in an underscore:


`binary_=True`	ask for the composite rendering, see Two wire formats
`counts_=True`	read what the statement changed, see What a write changed
`prepare_=`	override when psycopg prepares the statement
`row_=`	a psycopg row factory

Three methods run a statement, and each returns one kind of thing, so none of them returns a union: execute_query reads it all and gives a result, stream gives an iterator that keeps the rows on the server, and execute is psycopg's own and gives a cursor.

There is no graph_ and no timeout_, and both absences have a reason. Selecting a graph is a statement, is undone by a rollback, and drops the label table, so it belongs on the connection or on the pool, once, rather than once per statement. Bounding a statement's time is two more statements, one to set the limit and one to put it back, which measures over twice a bare RETURN 1 on a local server and is two more round trips against one a network away. A deadline therefore belongs where it is paid once for many statements: see Deadlines.

Graph values

(v,) = conn.execute_query("MATCH (n:Person) RETURN n LIMIT 1").records[0]

v.id                 # GraphId(labid=3, locid=1), prints as 3.1
v.label              # 'person', lower case: the server folded it, see Identifiers below
v.properties         # {'age': 42, 'name': 'Arthur'}, in the order the server stores them

(e,) = conn.execute_query("MATCH ()-[r:KNOWS]->() RETURN r LIMIT 1").records[0]
e.start, e.end       # the GraphIds of the two vertices

(p,) = conn.execute_query("MATCH p = (:Person)-[:KNOWS]->() RETURN p LIMIT 1").records[0]
p.vertices, p.edges  # in order
list(p)              # interleaved: vertex, edge, vertex, ...
len(p)               # element count
p.length             # hop count

A vertex and an edge are values rather than handles. Each is immutable, compares and hashes on its identity alone (which is what the server does too), and so can be a dictionary key or a set member.

A property map is decoded when you first read it, not when the row arrives. A query that ranks or counts vertices without looking inside them never pays for the map at all, and what that is worth grows with the map: reading five thousand vertices whose properties are never touched is about 1.2 times cheaper on a map of three keys, 1.5 times on a map of sixty, and 1.6 times on a 1536-dimension embedding. Those are medians over eleven interleaved passes; the map has to be worth something before skipping it is, which is why the narrow map barely moves.

It also means the map is decoded under whatever number setting is in force at the moment you touch it, rather than at the moment the row was fetched.

They are structs the garbage collector does not track, which is most of what makes a large result cheap: nothing here can take part in a reference cycle, since a property map holds only what JSON can. The public surface is read-only, and it is read-only by having no setters rather than by refusing writes; that is not only taste, because routing every field through a __setattr__ that raises costs more than storing the value does, on every element of every result.

label is the label a vertex was created with, and it is in the value the server sent. Its ancestry is not, and is not carried: labels(n) is a Cypher function the server answers with the row, own label first. Carrying it would put a label table lookup on the text rendering, which is the one reading that needs no table at all.

A row is a tuple, and the column names are read once per result into result.keys, so there is no dict per row and no name scanned per lookup. The same vertex appearing in many rows of a join is many objects that compare equal rather than one shared object; collapsing them would save one property decode per repeat, which on a join of a few hundred rows with kilobyte maps is under a hundredth of the read, against a pass over every result to discover whether there are any repeats at all.

Parameters

Placeholders are psycopg's %s. Cypher's own $n also works where the server accepts it.

conn.execute_query("MATCH (n:Person) WHERE n.name = %s RETURN n", ("Arthur",))
conn.execute_query("MATCH (n:Person) RETURN n LIMIT %s", (10,))
conn.execute_query("CREATE (:Person %s)", ({"name": "Arthur", "age": 42},))
conn.execute_query("MATCH (n) WHERE id(n) = %s RETURN n", (vertex.id,))

A Python str is sent as text. That matters more than it sounds. Left untyped, the server parses the bytes as JSON, so "Arthur" was an error and "123", "null", "true" and "1.5" were accepted and silently matched nothing. Declared as text, the server's own text to JSON coercion runs instead and every one of those becomes a correct answer. The plans and index conditions are unchanged by this, which was checked across equality, ranges, STARTS WITH, IN, ORDER BY and <>, including under a forced generic plan.

What it costs: a str passed where the type is not text-like no longer resolves, and says so while the statement is parsed, naming both types. The fix is psycopg's own documented idiom, %s::date. Passing the value's real type (date, UUID, int, Decimal, bytes) is untouched, and concat(%s, %s), which fails without this, works. To send a value the old way for one parameter:

from agensgraph import Unspecified
conn.execute_query("SELECT %s", (Unspecified("123"),))

A mapping is sent as jsonb, because psycopg cannot adapt a bare dict at all and nearly every parameter a Cypher statement takes is read as jsonb. A list stays a PostgreSQL array, because plain SQL on the same connection needs WHERE x = ANY(%s) to keep working.

Where a parameter is allowed

position	binds as
a general expression, `WHERE`, `RETURN`, list and map values, `ORDER BY`	jsonb
`LIMIT`, `SKIP`, `OFFSET`	bigint
a pattern property map, `SET a = %s`, `CREATE (:l %s)`	jsonb
`a[%s]`	jsonb
`id(a) = %s`	graphid
`size(%s)`	text (2.18 and later; earlier parsers take no parameter there)
a label, `a.%s`, `{%s: v}`, the lower bound of a walk	not allowed, syntax error
`UNWIND %s`, `x IN %s`	needs an explicit `::jsonb`

One shape is refused by the driver before it is sent. A parameter as the upper bound of a variable-length relationship, [r*1..%s], is not a syntax error: the server reads it as an unbounded walk plus a property map, prepares without complaint, and quietly returns the wrong rows. All three spellings of it are refused, in both placeholder styles:

conn.execute_query("MATCH (a)-[r*1..%s]->(b) RETURN a", (3,))   # ValueError

Sending a property map

Rendered with msgspec, which is where most of the cost of a write is: several times faster than the standard library on a small map, and about ten times on an embedding, where the numbers dominate.

NaN and the infinities are refused, wherever they appear. jsonb has no way to store one, and encoding it would write null instead, which is the wrong value rather than an error. Sent alone rather than in a map, a non-finite float reaches the server as a float8 and converting one to jsonb stores the text "NaN", so what came back would be a string where a number was sent. Both are refused.

Values that are rendered to rather than stored as what was sent:

written	stored as	reads back as
`datetime`, `date`, `time`	its ISO text	`str`
`UUID`	its text	`str`
`bytes`	base64 text	`str`
`set`	a JSON array	`list`

A Decimal is not in that table: it is stored as a JSON number and keeps every digit, because jsonb stores a number as numeric, which is arbitrary precision.

Identifiers

A label or a property key cannot be bound as a parameter, since the grammar has no place for one there, so a statement naming one dynamically has to carry it in its text. That quoting lives in one place:

from agensgraph.cypher import quote_identifier, quote_string

quote_identifier("person")      # person
quote_identifier("Person")      # "Person", since the server folds an unquoted name to lower case
quote_identifier("my label")    # "my label"
quote_identifier('a"b')         # "a""b"
quote_identifier("MATCH")       # "MATCH"

A name holding a null byte is refused rather than quoted, because the server's lexer stops at one and the statement would end somewhere other than where it appears to. Everything in this driver that builds a statement around a name (the index and constraint builders, the vector helpers, the identity map, LISTEN) goes through this.

Two wire formats

The driver never rewrites a query. RETURN n is sent as written, the server answers in its text rendering, and the driver reads that. It needs nothing from the server beyond the answer itself.

The same query can be asked for in the composite rendering instead, per-statement, with binary_=True. That form leaves the label name out of the value, so it needs a label table for the connection, which conn.graph(name) fills.

Ask for it when the result is large and the link is not the bottleneck. It trades bandwidth for parsing: every element repeats its column oids, its lengths and a tuple id the text form never writes, so a vertex is about a fifth more bytes and an edge several times more. In exchange the driver reads lengths instead of measuring where each value ends.

How much it pays depends on the shape, and only one shape pays enough to be worth quoting a number for. Over five thousand rows, measured interleaved across fifteen passes and reported as the range rather than a single figure, because the range is the useful part:

shape	slowest pass	median	fastest pass
edges	1.12	1.29	1.38
paths	0.85	1.02	1.09
whole vertices	0.82	1.17	1.66

So ask for it on edges, and measure before believing anything about the other two. Edges gain because an edge is mostly the four fixed-width fields the binary form states the lengths of. A path and a vertex are mostly a property map, and both renderings hand that to the same decoder, so what is left to win is small enough that run-to-run variation swamps it: the vertex figure moved by a factor of two across passes of the same measurement on one machine, and an independent run on another reported a median below 1. A number quoted to two digits for either would be noise presented as a measurement.

Embeddings are the one clear case, because a vector's text is decimal and its binary is the numbers themselves.

When in doubt, leave it off and turn it on for the one query that is slow.

Both renderings produce the same objects, and the suite asserts that against values the server itself produced, comparing them strictly rather than with ==, which would pass on disagreement and fail on agreement in five separate ways.

What a write changed

result = conn.execute_query("CREATE (:Person {name: 'a'})", counts_=True)
result.counts.inserted_vertices     # 1

Five counters: inserted_vertices, inserted_edges, deleted_vertices, deleted_edges, updated_properties.

A counter is None rather than 0 when it cannot be attributed to this statement, which is a real case rather than a formality. The counters live on the session, and the server zeroes them per clause group, so a write that returns rows leaves the groups its own clauses do not touch holding whatever an earlier write left there. The driver reads them before and after and reports only what it can account for: a counter that changed can only have been changed here, one that did not and was already zero is zero either way, and the one case where a stale number and a real one look alike is left unanswered instead of guessed.

That extra reading is only taken where it is needed. A write that returns no rows zeroes all five on the server, so one reading afterwards is the whole answer.

Streaming a large result

with conn.transaction():
    for (person,) in conn.stream("MATCH (n:Person) RETURN n"):
        ...

A server-side cursor, so the rows stay on the server. A hundred rows are fetched at a time by default, because the standard's own default of one is a round trip per row.

The grammar has no Cypher arm for DECLARE ... CURSOR, so the statement is wrapped in SELECT * FROM (...) t, and that wrap accepts only the read-only subset. A write is refused by name before anything is sent. A trailing LIMIT or ORDER BY is fine; one in the middle is not, and the server says so rather than the driver keeping a second copy of the server's grammar (the two releases disagree about that case, which is exactly why).

A transaction is required, and not as a formality: it is what makes abandoning the iterator safe, because leaving the transaction closes the cursor with it. Each stream names its own cursor, so two open at once do not collide.

Numbers in a property map

jsonb keeps an arbitrary precision decimal; Python's float does not. Integers of any length survive exactly. Only non-integers lose anything, past about seventeen significant digits.

1e400 is worth calling out: the server stores it as an exact 401 digit integer, not an infinity.

For the cases where the lost digits matter:

agensgraph.read_numbers_exactly()      # once, at startup

Every non-integer then reads as a Decimal, keeping whatever the server holds, 1e-400 included. What it costs depends on what the map holds, since only a non-integer takes the slower path: a map of integers is unchanged, a mixed one costs twice as much, one of non-integers about four times, and an embedding of a thousand floats closer to six. It applies to a property map and to a bare jsonb column alike, so RETURN n and RETURN n.p agree about the same stored value.

It is process-wide and meant to be chosen once at startup, which is also the only moment it is unambiguous: a map on the text path is decoded when it is first read, so two rows of one result can disagree if the setting changed between touching them.

Columnar export

from agensgraph.columnar import to_arrow, to_pandas, to_polars

table = to_arrow(conn.execute_query("MATCH (n:Person) RETURN n.name, n.age"))

Each column is built as a column, with its type declared rather than inferred, which measured about 3.3 times faster than pyarrow.Table.from_pylist over the same rows turned into dicts, building the identical schema. That is the comparison worth making: one that stringifies the values instead is not building the same table and can come out either way.

A whole vertex becomes a struct of its identity, its label and its property map, and the map is the JSON text taken from the bytes it arrived in, so it is never decoded into a dict:

table = to_arrow(conn.execute_query("MATCH (n:Person) RETURN n"))
# n: struct<id: uint64, label: dictionary<values=string, indices=int32, ordered=0>, properties: string>

An edge carries start and end as the same uint64 a vertex's id is, so a join between them is an integer join. A path is a struct of a list of vertices and a list of edges.

Layout decides the cases where more than one answer is defensible:

from agensgraph.columnar import Layout

Layout(elements="columns")            # spread a vertex over n.id, n.label, n.properties
Layout(identity="text")               # '3.1' rather than the packed integer
Layout(properties=some_struct_type)   # pull named fields out of the map
Layout(properties="skip")             # leave the map out
Layout(labels="text")                 # not dictionary encoded
Layout(vectors="list")                # not fixed-size
Layout(sparse="dense")                # expand a sparse vector

An embedding becomes FixedSizeList<float32> straight from the wire bytes, with no Python float in between. Ask for the composite rendering and it exports dramatically faster, in half the memory:

result = conn.execute_query("MATCH (n:Emb) RETURN n.v AS v", binary_=True)
table = to_arrow(result)          # v: fixed_size_list<item: float>[384]

A large result is exported a chunk at a time. batches() and reader() take a server-side cursor and never hold more than one chunk of Python objects. The first chunk settles the schema and every later one is held to it, so the chunks concatenate:

import pyarrow.dataset
from agensgraph import columnar

with conn.transaction(), conn.cursor(name="export") as cursor:
    cursor.execute('SELECT id, v FROM "graph".emb')
    pyarrow.dataset.write_dataset(columnar.reader(cursor, size=8192), "out", format="parquet")

columns(records, keys) is the same transposition without a backend, for a caller who wants plain lists. It keeps every column even when two share a name.

Awaiting the source. A cursor that has to be awaited needs the awaiting forms, since the blocking ones drain with a bare fetchmany and would be handed a coroutine:

from agensgraph.columnar import async_batches, async_to_arrow

async with conn.transaction(), conn.cursor(name="export") as cursor:
    await cursor.execute('SELECT id, v FROM "graph".emb')
    async for batch in async_batches(cursor, size=8192):
        writer.write_batch(batch)

table = await async_to_arrow(conn.stream("MATCH (n:Doc) RETURN n.v"), keys=["v"], size=8192)

async_to_pandas and async_to_polars match their blocking forms. A Result and a list of rows are sources here too, so a caller need not know which it holds.

There is no awaiting reader. A pyarrow.RecordBatchReader is pulled from rather than pushed to and refuses an awaiting iterator, so reading one from such a source means either holding the whole result (which is what a reader exists to avoid) or awaiting inside the pull, which stops the loop. Loop over async_batches and hand each batch to the writer instead, as above.

Bulk loading

conn.load_vertices("Doc", [{"key": "a", "title": "..."}, ...])
by_key = conn.identity_map("Doc", "key")
conn.load_edges("Cites", [(by_key["a"], by_key["b"], {"weight": 1})])

COPY in binary rather than a statement per row: about thirty times a statement at a time, and better than half again on top of the best a single UNWIND ... CREATE can do.

No identity is supplied. The label table's id column has a default that builds the graph id from the label's own id and its sequence, so copying only the property map produces exactly the identities a CREATE would have. That removes the whole business of generating identities client-side and keeping them unique.

Edges need the identities of the two vertices they join, which is what identity_map reads. It raises rather than guessing if the key is not unique, or if an element does not have it, because silently collapsing two vertices into one would attach every edge of both to whichever survived.

Name the keys you need, if you know them:

conn.identity_map("Doc", "key")                 # the whole label
conn.identity_map("Doc", "key", keys=wanted)    # only these

Reading the whole label is not something an index can make cheaper, and that is worth knowing rather than discovering. Every property of an element lives in one jsonb column, so reading one key reassembles all of them; and PostgreSQL will not answer a projection from an index over an expression, so even a purpose-built index on the same expression goes unused with sequential scans turned off. On 20,000 elements each carrying a 1536-dimension embedding the whole label costs about 670 ms whatever is indexed, against 11.7 ms for a thousand named keys.

Either spelling of a key finds it, so this agrees with the whole-label read: a key stored as the number 1 and one given as "1" are the same key. With nothing to look a key up by, naming them falls back to reading the label, since a lookup would read it once per key.

Writing what may already be there

load_vertices is a copy, and a copy only creates, so reading a source in twice makes a second element for everything the graph already holds. upsert_vertices reads which keys are there, copies only the rows whose key is not, and leaves the rest alone:

conn.upsert_vertices("doc", "key", rows)                          # insert what is missing
conn.upsert_vertices("doc", "key", rows, on_existing="update")    # and refresh the rest

Measured on 20,000 rows with half already present: about fourteen times a MERGE per row, the route every knowledge-graph package takes. "skip" is the default because it is a copy and nothing else; "update" adds one statement for the whole overlap, addressed by the identities the read already found, which plans as an index probe per row rather than a scan. It merges rather than replaces, so a property you do not mention keeps its value.

A key with no uniqueness behind it is refused. This is not caution: eight writers merging on twenty-five shared keys, with only a plain index, produced thirty-two elements for twenty-five keys on a live server. With a unique index it produced twenty-five, raising 23505 and 40001 along the way, both of which are the expected path. require_unique=False accepts the hazard for a graph that cannot add one.

An edge is upserted by the pair it joins, since that pair is what an edge is:

conn.upsert_edges("links", [(start_id, end_id, {"weight": 3}), ...])
conn.upsert_edges("links", pairs, on_existing="update")

Measured on 2,000 edges: about sixteen times a MERGE per edge, and 1.45 times a plain load_edges, that difference being what not creating a duplicate costs. A pair given twice in one call is written once, because the copy reads nothing back and the repeat would be the second edge this exists to prevent.

A label with nothing keeping one edge per pair is refused, for the same reason a vertex key is: eight writers over twenty-five pairs produced twenty-seven edges and reported no failure at all. The index that prevents it has to be written as plain SQL, because a property index cannot key on the endpoint columns (see above):

conn.execute(f'CREATE UNIQUE INDEX links_pair ON "{graph}".links (start, "end")')

With it in place the same eight writers left twenty-five edges. require_unique=False accepts the hazard for a graph that cannot add the index.

Which read it uses depends on the batch. Asking about the pairs sends two identities each and reading the label sends none, so asking wins until the batch is as large as the label: measured against 20,000 edges, asking took 3.3 ms for 100 pairs and 8.0 for 1,000 against the 53.7 the label read costs whatever is asked, and lost 143.7 to 53.7 at 20,000. The label's own size decides, taken from the planner's estimate rather than a count, and read in the same statement that checks the index, so neither costs a round trip of its own. A label nobody has analysed reports no size and is asked about.

The keys are looked up by name rather than read out of the label, so the cost follows the batch and not the graph. Both spellings of each key are asked for, because a key stored as the number 1 and one given as "1" are the same key here, and a lookup by property alone would treat them as two.

Give an ingest key a column of its own

If a label is written to more than once, declare its key as a promoted column. This matters more than anything the driver does, and it is a schema decision rather than a driver setting:

conn.execute("CREATE VLABEL doc (key text generated)")
conn.execute("CREATE UNIQUE PROPERTY INDEX ON doc (key)")

All of a label's properties live in one jsonb column, so reading any single key reassembles the whole map. On a label whose elements carry a 1536-dimension embedding that is expensive out of proportion to the key: the heap is small and the TOAST table is enormous, and the planner cannot see the second one at all, so it will happily choose to scan.

A promoted key is a separate column, so reading it never touches the map. Measured on 20,000 elements each carrying a 1536-dimension embedding, reading every key:


from the property map	731 ms, 60,183 buffers
from a promoted column	3 ms, 163 buffers

It is also the only form that reaches an Index Only Scan. An index over a property in the map holds it as jsonb and the planner will not answer a projection from such an index, so even a perfectly matched index still visits the heap and still reassembles the map.

Two things to know before promoting one. Adding the column to a label that already has elements rewrites the table, which reads every map once. And a whole-element read (RETURN n, or writing the element) still goes through the map by design, so promotion helps reads of that key, not everything.

A frame goes in directly, without becoming Python objects on the way:

conn.load_vertex_frame("Person", table)      # each column is a property
conn.load_edge_frame("KNOWS", edges)         # start and end are packed identities

That is about half again faster than the same table handed to load_vertices() as mappings, and no more than that, because almost all of a load is the server's own ingest. The client's share is what falls, by about eight times.

A burst of statements

For a batch whose cost is round trips rather than work:

conn.pipeline_batch([f"CREATE (:Event {{n: {n}}})" for n in range(1000)])

A failure is attributed to the batch, not to a statement, and the signature says so. That is not caution, it is what the server does: with four statements where only the second is bad, the first raised and the rest reported no SQLSTATE at all. BatchFailed carries the statements that were sent, in order, and the server's own error as its cause. Re run them one at a time to find out which.

Reading the catalogs

conn.graphs()                    # [Graph(name, schema, labels)]
conn.labels()                    # [Label(id, name, kind, parent)]
conn.declared_properties()       # [DeclaredProperty(label, name, type, nullable)]
conn.indexes()                   # [Index(label, name, unique, definition)]
conn.constraints()               # [Constraint(label, name, unique, definition)]
conn.element_counts()            # {'Person': 2, 'KNOWS': 1}

Each takes an optional label to narrow to, and an optional graph to read about instead of the one the session is on.

Counting per label reads no property: the label id is part of every element's identity, so the group key is a function of the id column alone. Whether the heap is read anyway is the planner's to decide, and often yes, because against a narrow label it prefers a sequential scan to an index only one. An edge can never avoid it, because the engine creates an edge's id index as BRIN, which carries no tuple pointers.

Two things about uniqueness are worth knowing, because they are not where you would look. A uniqueness constraint in AgensGraph is an exclusion constraint, so it is filtered out of the property index view, and a driver reporting from that view alone would show a graph as having no uniqueness constraints when it has them. constraints() reads the constraint catalog as well, so it finds them.

declared_properties() asks the server whether it can promote a property at all before reading the catalog that records one, because that catalog does not exist on every server and the version cannot tell you: the 2.18 release branch and main both report 2.18-devel, and only one of them has it.

Describing a graph

For a prompt, a schema browser, or anything that has to say what a graph looks like:

description = conn.describe()

description.labels        # name, kind, and what each inherits
description.properties    # {label: (PropertyShape(name, kind, declared), ...)}
description.triples       # (Triple(start, edge, end, edge_count), ...)
description.counts        # {label: how many}
description.meta_gathered # whether the triple catalog has ever been filled

Nothing here scans the graph. Labels, counts and triples come from catalogs; only the property names look at rows, and at sample=100 of them per label rather than all. Measured against the three full scans this replaces, on 60,000 vertices and 120,000 edges: 55 times, and the gap widens with the graph, because the scans grow and this does not.

It installs nothing. Three of the integrations it replaces create a plpgsql typeof(jsonb) in your database. jsonb_typeof is built in, and the one thing it does not do, telling a whole number from a fractional one, is done in Python. The tests assert pg_proc and pg_class are the same size afterwards.

Where a property has a column of its own its type is read from the catalog and declared is true, which is exact; everything else is what the sample held. A label whose name has a space is found, and so are edge properties and label inheritance, all three of which a reading of the text loses.

The triple catalog is filled by a gather, and auto_gather_graphmeta is off by default, so on a server where nobody has gathered there are no triples and meta_gathered is false. That is reported rather than guessed at: no triples and no edges agree with each other, and no triples with edges present does not.

conn.describe(refresh=True)     # gather first, which is a write

Declaring what should exist

Say what the schema should be, and let the driver work out the difference:

from agensgraph import DesiredIndex, DesiredLabel, Unique, Check

conn.ensure_labels([
    DesiredLabel("person"),
    DesiredLabel("knows", kind="e"),
    DesiredLabel("employee", parent="person"),
])
conn.ensure_indexes([
    DesiredIndex("person", ["email"], unique=True),
    DesiredIndex("doc", ["title"], method="gin"),
])
conn.ensure_constraints([
    Unique("person", "email"),
    Check("person", "age > 0", "person_age_positive"),   # a check needs a name of its own
])

An edge's endpoints cannot be keyed through DesiredIndex, and asking is refused. start and end are columns of the edge's own table, while a property index keys on the property of the name it is given. So a unique index naming them is built over properties.'start' and properties.'end', an edge carrying no such property indexes NULL, and no two NULLs conflict: measured before this was refused, the index was accepted, reported as unique, a duplicate edge was still taken, and eight concurrent merges of one triple left two edges. The server prints it as (start, "end"), exactly as the columns would print, so nothing afterwards could tell them apart.

For that guarantee, write it as plain SQL over the columns, which does refuse a duplicate with 23505:

conn.execute(f'CREATE UNIQUE INDEX links_pair ON "{graph}".links (start, "end")')

It is not reconciled, for the same reason indexes() does not report it: that reader admits an expression index or one over a promoted column, which are the server's own conditions, and endpoint columns are neither.

Declare the labels a server writes to. A write to a label that is not there makes one, and that is DDL inside whatever transaction the write is in, so two writers arriving together race. Measured with eight writers merging one edge of an undeclared label: seven of the eight failed, six with 23505 and one with 42P07. With the label declared first, none did.

ensure_labels never drops, whatever is left out of the list. A label holds the elements written to it, so removing one is a decision about data rather than schema, and a reconciler able to take it could empty a graph by being handed a shorter list. A name already there as the other kind is refused, naming both kinds.

All three return the statements they ran, take dry_run=True to return them without running any, and drop_extra=True to remove what is there and not asked for. Running the same declaration twice is a no-op the second time.

DesiredIndex and Unique derive a name when you do not give one, from the label and the properties. Check cannot, since an expression gives nothing to derive from, so its name is required.

Name a label the way the server stores it, which conn.labels() will tell you. Every name here is quoted, so it is taken literally: CREATE VLABEL Person stores person, and asking for DesiredIndex("Person", ...) then looks for a label that does not exist. Write "person", or create the label quoted in the first place so the capital survives. This is the one place the folding rule in Identifiers reaches out and bites, because everywhere else both sides fold together and agree.

The matching is deliberately conservative, because of how the server stores these. A name is derived from the columns, truncated, then given a counter on collision, so names are not the key. A definition is printed with defaults omitted. A predicate is stored normalised, so age > 0 comes back as something that does not look like what you wrote. Three consequences: name an operator class only when it differs from the default, expect a partial index to be compared on its normalised predicate, and let anything the reconciler cannot match confidently alone rather than dropping it.

Embedding vectors

Needs pgvector in the database. It is created per database, not per server, so ask the connection rather than the version:

conn.has_vectors()          # is the extension created here
conn.register_vectors()     # read vector, halfvec and sparsevec on this connection
conn.vector_version()       # (0, 8, 6)

Two routes, and both are indexed the same way:

# a property left in the map, with the cast carrying the dimension
conn.execute("CREATE PROPERTY INDEX ON movie USING hnsw ((embedding::vector(4)) vector_cosine_ops)")

# or a column of its own, which needs no cast and refuses a wrong-length value on write
from agensgraph.vector import generated_column
conn.execute(f"CREATE VLABEL doc ({generated_column('embedding', 1024)})")

A promoted column is the stronger of the two: the dimension lives on the column, so vector_in enforces it at the moment a value is written rather than at search time.

Storing and reading

from agensgraph.vector import Vector

conn.execute("CREATE (:doc {embedding: %s})", (Vector(values),))

(v,) = conn.execute_query("MATCH (n:doc) RETURN n.embedding").records[0]
len(v)              # free, the dimension is in the first two bytes
v.values            # array('f'), which numpy and torch take without copying
v.tolist()          # an ordinary list
v[0], v[-1], v[0:2], list(v), sum(v), 2.0 in v, v.index(2.0)
v == [1.0, 2.0]     # True when the numbers match

A Vector keeps the bytes the server sent and turns them into numbers only if you look. That is the same bargain the driver makes with a property map, and it pays here for the same reason: a vector search asks the server for the distance, so the components of the vectors it ranked are often never read. It is not a list (isinstance(v, list) is False, json.dumps(v) raises) but it compares equal to one, which is the part that would otherwise go wrong quietly.

Ask for binary_=True when you read vectors. This is the one place the rendering makes a large difference rather than a small one: a vector of 1536 dimensions prints as roughly fifteen kilobytes of decimal against six of wire bytes, so the text form costs about three times as much before anything is parsed and seven times once the numbers are read.

Send a Vector rather than a list. A list is formatted a number at a time as decimal for the server to parse back, and measured over eight hundred writes of a 1536-dimension vector into a promoted column it costs 2.3 times the Vector route. A string you formatted yourself costs about the same as a Vector, having already paid the formatting once, but it is still the worse choice, because nothing checks its dimension or its numbers until the server does.

The ::vector(n) cast belongs on the reading side, in the search and in the index expression above. There is no reason to write one into a property: the column's own type settles what a promoted property is, and a property left in the map is jsonb whatever you cast on the way in.

In bulk it is the other way round. load_vertices writes a property map as JSON and a Vector is deliberately not JSON encodable, so passing one raises Encoding objects of type Vector is unsupported. Bulk-load vectors as plain lists, and give the property a column of its own to get them back as Vector: measured, a list loaded through load_vertices into a vector(1536) generated column reads back as a Vector.

Half precision is halfvec, and works the same way. Binary quantisation is available in SQL, where bit is spellable; Cypher has no syntax for that cast, which is worth knowing before you look for one.

Indexing and searching

The index statement and the search statement come from the same function, so they cannot disagree:

from agensgraph.vector import vector_index, nearest, Distance

conn.execute(vector_index("doc", "embedding", dimensions=1024,
                          operator_class=Distance.COSINE.operator_class))
# create property index on doc using hnsw ((embedding::vector(1024)) vector_cosine_ops)

rows = conn.execute_query(
    nearest("doc", "embedding", dimensions=1024, operator=Distance.COSINE, limit=10),
    (query_vector,),
).records
# match (n:doc) return n order by n.embedding::vector(1024) <=> %s::vector(1024) limit 10

Distance is a StrEnum whose value is the operator, so it goes straight into operator=, and .operator_class gives the class the matching index needs.

That sharing exists because of two ways to lose the index silently:

The typmod must match exactly. Against a vector(4) index, a search casting to a bare vector or to vector(3) sorts a sequential scan and tells you nothing.
An operator class serves one operator. A vector_cosine_ops index answers <=> alone, and ordering by <-> against it scans.

Distances by name, because two of them differ by a single character and mean entirely different things: L2 (<->), INNER_PRODUCT (<#>, negated so that smaller is nearer as it is for the others), COSINE (<=>), L1 (<+>), and for bit strings HAMMING (<~>) and JACCARD (<%>). hnsw and ivfflat are both available, and vector_index takes options= for things like WITH (lists=10).

Writing the search by hand is fine too, and on an unpromoted property the cast is required rather than optional, since jsonb <-> vector is not an operator:

conn.execute_query(
    "MATCH (m:movie) RETURN m ORDER BY m.embedding::vector(4) <=> %s::vector(4) LIMIT 4",
    (query_vector,),
)

Sparse vectors

from agensgraph.vector import SparseVector

v = SparseVector({0: 1.0, 3: 2.0}, 6)   # indices, then how many dimensions
v.to_dense()                            # when you really want the zeros

It is a value of its own rather than a dense list, and the reason is size: three non zeros in a million dimensions is a few dozen bytes on the wire against roughly eight mebibytes as a list of floats, and expanding it would discard the entire reason the type exists.

Indices are zero-based in Python, everywhere, including to_dense(). The wire's text form is one based and its binary form is zero-based, which is a genuine trap: two renderings that disagree about the base, decoded without accounting for it, give a plausible off-by-one rather than an error. The conversion happens in one place, on the text boundary only, and is tested from both directions.

What the server refuses, the constructor refuses: duplicate indices, an index out of range, a dimension below one, a non-finite value. An explicit zero is dropped, because the server drops it, and mirroring that is what keeps a round trip stable.

A Cypher list cannot be written into a sparse column at all, so the dumper is not a convenience here; it is the reasonable way in.

Tuning a search

conn.vector_search_options({"hnsw.ef_search": 100})

Seven settings, listed with the type each takes in agensgraph.vector.SEARCH_OPTIONS. Applied to the transaction by default, so they do not leak into the next caller on a pooled connection; pass local=False to set them for the session.

Pooling

pool = agensgraph.ConnectionPool("host=localhost dbname=graph", graph="social",
                                 min_size=4, max_size=16)
pool.open(wait=True)          # fails here if the server is wrong, not later under load

with pool.connection(deadline=agensgraph.Deadline(5.0)) as conn:
    result = conn.execute_query("MATCH (n:Person) RETURN n")

pool.close()

It wraps psycopg's pool rather than reimplementing one, and adds five things psycopg has no equivalent for.

A server this driver cannot read is refused by open() before any worker starts. That check is its own connection, made and closed first, because a refusal raised inside the pool would be counted as a connection error and retried for the whole reconnect timeout instead of reported.

invalidate() retires every connection now held, in one call, and returns the generation now in force. Nothing is closed and nothing waits: a connection in use stays usable until its holder is finished, and is closed when it comes back. So a server restart costs one call rather than one failure per connection.

drain() is the other half and does a different job. It closes every connection and opens the same number again, for a change a connection carries from the moment it is made: a registration on the adapters map, a setting asked for in configure, a rotated password behind a callable connection string. The generation is untouched, because those connections are not wrong to be reused; they are replaced so the replacements are built the new way.

configure runs once per new connection and setup once per handout. psycopg has only the first, and a graph or a statement timeout belongs to the second.

A connection whose statement was interrupted is closed rather than lent again. psycopg does try to leave it clean, and when that finishes the connection really is reusable, but it cannot promise it finished: the read it re-enters is bounded by nothing, and because an asyncio timeout works by cancelling, no timeout on that connection can fire either. Such a connection is replaced, which costs one connection on an event that is rare by definition. A statement that merely failed keeps its connection, since a value the driver refused never reached the socket and a server's own refusal leaves the connection idle and answering.

get_stats() returns psycopg's own counters plus generation, connections_retired and connections_interrupted; pop_stats() is the same with the accumulating ones reset, for reporting an interval. resize() and check() pass through.

Starvation shows up as requests_wait_ms against requests_queued, which is how long borrowers spent waiting and how many had to wait at all. Measured with six slow borrowers against a pool of two: 8,846 ms of waiting across five queued requests, while a trivial query took 2.81 seconds just to get a connection. A pool that is too small looks exactly like a database that is slow until you read those two.

Every counter is always present, at zero when nothing has moved it. psycopg keeps them in a Counter, so a key exists only once something has incremented it and pop_stats() takes them all away again; reading one on a timer therefore raised KeyError on the first interval, on any quiet interval, and after every pop. agensgraph.pool.COUNTERS names the ten that accumulate, so an exporter can iterate them rather than hard-coding a list.

For a process that handles one request and exits, or a serverless one that may be frozen between requests, NullConnectionPool keeps nothing: one connection per caller, closed when they are done. Everything else is the same, so moving between the two is a change of class and nothing else. A borrow and one statement costs about eight times what it costs from a pool that keeps its connections, which is what connecting costs and is the reason not to choose it for a process that lives long enough to reuse one.

Both pools have an awaiting twin: AsyncConnectionPool and AsyncNullConnectionPool.

Deadlines

from agensgraph import Deadline

budget = Deadline(5.0)
with pool.connection(deadline=budget) as conn:
    conn.execute_query("MATCH (n) RETURN count(n)")

One budget, threaded through the wait for a connection and the statement that runs on it, so time spent waiting is time the statement no longer has. It reaches the server as statement_timeout, set once per borrow and reset when the connection goes back, so it cannot be inherited by the next caller.

statement_timeout is deliberately set a little inside the budget. Arriving at COMMIT with no time left manufactures the one failure that cannot be retried safely, so the gap is reserved rather than spent.

It never uses transaction_timeout or idle_in_transaction_session_timeout as a query deadline, because those terminate the session. Expired is a TimeoutError, so a caller that already catches one keeps working, and running out of your own budget is classified differently from a pool with nothing to give.

Retrying

RetryPolicy decides; it does not drive. You keep the loop, because only you know what is safe to run again and whether the transaction had already written:

import time
from agensgraph import RetryPolicy

policy = RetryPolicy(attempts=3)     # the first try counts, so this is one try and two retries
previous = []

for number in range(1, policy.attempts + 1):
    try:
        result = pool.execute_query("MATCH (n) RETURN count(n)")
        policy.succeeded()           # pays back into the shared allowance
        break
    except Exception as exc:
        attempt = policy.decide(exc, number=number)   # .retry .delay .recovery .reason
        if not attempt.retry:
            raise
        previous.append(exc)
        time.sleep(attempt.delay)
else:
    raise policy.exhausted(previous[-1], attempts=policy.attempts, previous=previous)

decide() also takes wrote=True, which says the transaction had written something, and that is what turns a lost connection from "reconnect and try again" into "the outcome is now unknown", and remaining= from your budget, so a wait that would not leave time for another attempt is not taken.

Retryability is six categories with a recovery action, not a boolean, because a boolean gets 26000 catastrophically wrong: reconnecting appears to fix it while hiding a broken reset hook that keeps firing.


`SAFE`	retry on the same connection. `40001`, `40P01`. The common case for a graph write.
`RECONNECT`	new connection, then retry. Class 08, `57P01`, `57P02`, `57P03`, `25P03`
`BACKPRESSURE`	retry, but wait longer. `53300`, `53200`, `55P03`, `55006`
`RESET_STATE`	clear the statement cache, keep the connection. `26000`
`UNKNOWN`	resolve it, do not guess. `08007`, `40003`
`FATAL`	never retry. `57014`, classes 42, 22, 23

Concurrent graph writes surface as 40001 by design, so a conflict is the expected path rather than an exotic one, and it is safe to retry.

Backoff is full jitter, capped before the jitter is drawn rather than after, which is a distinction worth keeping: capping afterwards makes the cap reachable only by chance and the distribution not the one the name describes. A backpressure retry doubles the ceiling before capping, so it waits longer without breaking the cap.

A counter is not a budget. Four layers each willing to try three times is eighty one attempts from one user action, so the allowance is a token bucket shared across the process: a transient failure costs more than a rejection does, because a connection failure is usually the whole service rather than this one request. When it is exhausted, the message says so, which saves a great deal of support time.

The allowance fills with time, and you should not rely on succeeded() to fill it. Four transient failures spend enough to stop the retrying, and calling succeeded() is the only other thing that pays a token back. Since the default allowance is one object for the whole process, an allowance spent and never credited used to mean every retry everywhere was off until a restart. Time returns the tokens now: an allowance drained to nothing is retrying again about a minute later, which is the timescale of a failover rather than of a request. Reporting successes still helps, and recovers it sooner. TokenBucket(capacity, refill=...) sets the rate, and a refill of zero is refused rather than accepted as an allowance that can only ever be spent.

The connection's fate is decided in one place on every exit path, including cancellation, rather than delegated to a callback that some paths skip.

A budget on a connection nobody pooled. A pool sets a limit per borrow; for a server the unit is a request, however many statements the request turns out to need:

async with conn.deadline(5.0) as budget:
    await conn.execute_query(one)
    await conn.execute_query(two)
    left = budget.remaining()

The limit is the server's, set below what the caller is waiting for, so the server gives up and reports a cancelled statement rather than the caller giving up first and leaving one running on a connection it has stopped reading. It is put back on the way out, including after a statement raised, and put back by name: a connection carrying options=-c statement_timeout=... returns to that rather than to none.

It is a block and not an argument to execute_query, and the reason is measured. Setting and restoring the limit is two statements: over a loopback connection select 1 alone is 95 microseconds, alone inside a block 303 (3.2x), and ten statements in one block 1.26x. So the cost is real and belongs where many statements amortise it. A budget with no limit sets nothing and costs nothing.

The two are sent as they read rather than in one pipeline. Over a loopback connection a pipeline of three costs 681 microseconds against 432, and one statement inside a pipeline costs 307 against 123 outside it: such a round trip is libpq and psycopg at both ends rather than latency, so a pipeline has little waiting to remove and its own cost per statement to add. Thirty statements come about even. What it costs where a round trip is latency is untested here.

Losing a connection mid-commit

The one failure that cannot be retried blindly is a commit whose acknowledgement never arrived: the write may have landed or may not, and doing it again could apply it twice.

with conn.transaction():
    xid = conn.transaction_id(assign=True)     # keep this
    conn.execute_query("CREATE (:Order {id: 1})")

# if the connection is lost while committing, ask a different connection what became of it
outcome = other_conn.resolve_commit(xid)

CommitOutcome.COMMITTED means it landed, so do not retry. ABORTED means it did not, so retry as you would a conflict. IN_PROGRESS means back off and ask again. UNKNOWN means the server can no longer tell, which is not the same as aborted and has to be surfaced rather than guessed either way.

Stashing the id is nearly free, since any CREATE assigns one anyway.

Cancellation

To stop a statement that is still running, ask the server to stop it:

await conn.cancel_safe(timeout=5.0)     # psycopg's, and it is the one to use

The statement's own caller then receives 57014, which is the only evidence the cancel landed. It uses libpq 17's cancel connection where there is one and falls back to the older request otherwise. Measured against pg_sleep(20): the call returned in under 10 ms and the backend went from active to idle.

A connection interrupted mid-statement is discarded; one cancelled cleanly is kept. Those are different things and the difference is worth a sentence. A task cancelled while awaiting a statement leaves a connection whose state nobody can describe, and psycopg's own recovery re-enters a wait bounded by nothing, so such a connection is replaced rather than reasoned about. A cancel_safe that ends in 57014 is not that: the statement finished, the protocol is in step, and the connection is idle. Measured through a pool of one, the first is handed a new backend and the second is handed the same one back.

Discarding the interrupted one is the majority position among drivers, and the alternatives have a bad record: the two cleverer attempts in one widely used client produced a CVE each, one of which could send one request's response to a different request. The rule here is to decide the connection's fate synchronously first, with no awaits, and only then bound and isolate any cleanup.

Abandoning a task is not cancelling it. A statement whose task is never awaited and never cancelled goes on running: measured, the backend stayed active until it was terminated by hand. Nothing in a driver can see that, since from here it is a statement that has not answered yet. Cancel the task or call cancel_safe; do not simply stop waiting.

57014 has four separate causes (a client cancel, statement_timeout, a recovery conflict, and a tie broken toward lock timeout) and its message is localised, so the driver tracks "I cancelled this" in its own state and never parses message text. A successful cancel dispatch is not proof either: the request matches on process id and key and returns silently on a mismatch, and process ids are reused. The only evidence a cancel landed is 57014 on the original connection.

When the network goes quiet

Keepalive is asked for by default: without it, a connection whose network stops carrying packets waits for the kernel, which is two hours and a quarter on Linux.

Each setting is filled in on its own, so naming one does not silently leave the others at the system's values. keepalives=0 turns them all off.

tcp_user_timeout is not among them, and that is worth saying, because it is the setting usually recommended for this. It bounds how long transmitted data may go unacknowledged, and a connection waiting for a reply has transmitted nothing. Measured against a server whose traffic was dropped:

	outcome
nothing set	still waiting when the test gave up
`tcp_user_timeout` alone	still waiting when the test gave up
keepalive, with or without `tcp_user_timeout`	failed within seconds

So it is useful alongside keepalive and useless instead of it. It is left unset because it also applies while sending, where too small a value would end a healthy connection.

Observability

def log(record):
    print(record.statement, record.elapsed, record.rows, record.failed)

agensgraph.add_query_logger(log)
agensgraph.enable_tracing()      # needs agensgraph-python[otel]

Every statement is reported, not only the ones execute_query sent: conn.execute, a cursor you took yourself, and the driver's own catalog reads all arrive at the same place, because a caller asking what the driver sends wants the round trips they did not write. elapsed is the round trip, sending the statement until the server has answered, and not the reading of the rows afterwards.

Off costs a boolean test. On costs less than this can be measured: over four hundred statements a run, attached measured 4% faster than detached, while two detached runs differed from each other by 3%, so the effect is under the noise rather than small. Take the direction and not a figure. A clock is not read, and a timer is not even allocated, unless something is going to ask for the number. Spans are per statement and never per row, the tracing API is imported only when asked for and the SDK never, the record carries an opaque connection number rather than the connection's settings (which hold the password), and no parameter value ever reaches a span.

Notices are structured, drained per-statement so they do not leak into the next result:

agensgraph.observability.add_notice_listener(lambda n: print(n.severity, n.message))

A callback runs on whatever ran the statement, both of these, and it is called rather than scheduled. On the awaiting interface that is the task the statement is on, so a callback that blocks stops the event loop for as long as it blocks. Keep them to putting a value somewhere: append to a list, increment a counter, put onto a queue that something else drains. Writing to a socket or a file belongs on the other side of that queue.

Raising in one is safe: it is logged and swallowed, so a listener cannot replace the statement's own outcome, and one listener raising does not stop the others being told.

Handing a result to something that wants JSON

agensgraph.to_json(result.records)      # bytes, and the fast one
agensgraph.to_builtins(vertex)          # dicts and lists
json.dumps(vertex, default=agensgraph.json_default)

The shape is the wire's:

{"id": "3.1", "label": "person", "properties": {"name": "Arthur"}}
{"id": "4.1", "label": "knows", "start": "3.1", "end": "3.2", "properties": {}}
{"vertices": [...], "edges": [...]}

An identity is the text the server prints, "3.1", which goes back through %s::graphid unchanged. The structured {labid, locid} was considered and dropped: it invites a reader to build an identity out of two numbers, and one built that way has never been near the server.

to_json does not decode a property map nobody read. It copies the bytes the map arrived in straight into the output, which is the same bargain the driver makes everywhere else and it pays here most of all: on a result of 1536-dimension embeddings it is hundreds of times faster than the route through json.dumps, which decodes every number into a Python float and writes it out again. On small maps it is about twice.

Do not encode a graph value directly. msgspec.json.encode(vertex) emits the private fields, so _id arrives as a nested {"labid": 3, "locid": 1} next to _raw and _props. There is no encoder hook that prevents it, either: a hook runs only for a type the encoder does not already know, and a vertex is a msgspec.Struct, so it never fires. The conversion has to happen before the encoder sees the value, which is what these three do.

Running a statement you did not write

Model output, most often. The rule is that the server refuses a write rather than the driver, because a driver that reads the text to decide has to recognise every way of writing, and does not: this is PostgreSQL underneath, so INSERT, TRUNCATE and DROP are all available and none of them is Cypher.

with conn.read_only_transaction():
    result = conn.execute_query(whatever_the_model_said)

Measured inside that transaction: every Cypher write, INSERT, TRUNCATE, DROP and CREATE VLABEL are refused with 25006, and a plain read runs. The transaction characteristic is psycopg's own connection.read_only, which is the thing to set for a whole session; what this adds is the check below.

What a read-only transaction does not stop, and it is worth knowing. COPY ... TO PROGRAM runs a command on the server's host and is allowed, because it takes rows out of the database rather than putting any in, so there is no write to refuse. Reading the statement does not stop it either: a second statement after a semicolon, and a leading comment, both get one past. What stops it is not holding the privilege, so that is what is asked about:

conn.can_run_server_programs()      # asked once per connection, and kept

A role that holds it is refused the transaction, with a message saying what to change, rather than left to find out. Pass allow_server_programs=True to accept it. Connect as a role that is neither a superuser nor a member of pg_execute_server_program, and the boundary holds:

	refused by
a Cypher write, `INSERT`, `TRUNCATE`, `DROP`, DDL	the transaction, `25006`
`COPY ... TO PROGRAM`, including one hidden behind another statement	the privilege, `42501`
a plain read	nothing, it runs

One thing the server will not refuse for you: a second statement. Text sent with no parameters goes over the simple query protocol, which runs every statement in the string and reports only the first one's result, so a read with a write after a semicolon runs the write and looks like the read. Measured: select 1; create table t(i int) returned a row and left the table behind.

cypher.check_single_statement(model_output)     # refuses, naming the word that is the reason

Binding a parameter, any parameter, closes the same hole from the other side: it moves the statement onto the extended protocol, where the server does refuse it with 42601. Callers taking text from somewhere else want both. The check reads the text with its literals blanked, so a semicolon inside a string, a dollar-quoted body or a comment separates nothing, and a terminating semicolon is not a second statement.

cypher.check_can_wrap() is a reasonable hint to show a user early, and it already beats a write-detecting regex on RETURN n.set. It is not a boundary and is not used as one.

Errors

They are psycopg's exception classes, raised by psycopg, so except psycopg.errors.UniqueViolation keeps working and no wrapping layer stands between you and them. The engine mints no SQLSTATE of its own, which was checked against its source rather than assumed, so there is nothing to register and nothing that reaches into psycopg's global tables for unrelated connections.

Three of the server's refusals cannot be told apart by SQLSTATE, and are translated into named classes so you do not have to match on message text:


`ConfigurationError`	`enable_graph_dml` or `enable_eager` is off. Both arrive as `XX000`
`ReadOnlyGraphWrite`	a graph write in a read-only transaction, which the server reports as `cannot execute ??? ...`
`CapabilityError`	a feature this server does not have, naming the version that does

Plus the driver's own: BatchFailed, StaleLabelCache, ReleasedConnection, StaleGeneration, UnresolvedCommit, NoEnclosingTransaction, Expired.

What a failure prints as

PostgreSQL puts row data in a failure's DETAIL. A uniqueness failure reads Key (email)=(alice@example.com) already exists, so a plain logger.exception() writes into the log a value the driver never saw as a parameter. DETAIL and CONTEXT are therefore not part of the message by default. What the message loses, exc.diag.message_detail and exc.diag.context still hold, so the data stays for a post-mortem and only the rendering is cut.

agensgraph.show_error_details(True)   # process-wide, so it cannot be forgotten at a call site

It cuts DETAIL and CONTEXT and nothing else. A primary message can still echo the value it was handed, because that is the server quoting its own input rather than a value belonging to another row.

Handing a failure to somebody who did not send the statement wants the SQLSTATE and the primary message and nothing else, which safe_message is:

agensgraph.safe_message(exc)     # '23505: duplicate key value violates unique constraint "doc_email"'

A model reading its own error is the case for it, and so are a log line, an API body and a tool result. It never reaches DETAIL, it stays one line, show_error_details does not affect it, and it answers for a failure the driver raised itself, which carries no SQLSTATE, as well as for one psycopg raised.

A copy's failure is described the same way. It arrives when the copy's block ends rather than from a statement, so the cursor that describes every statement never saw it: load_vertices of a duplicate key put the conflicting value into str(exc) where the same write through a statement did not. All four copy paths now read the same as a statement, which is asserted by comparing the two messages.

agensgraph.errors.mask_dsn() is there for a caller who holds a connection string and wants to write it somewhere. Nothing in this driver calls it, and that was checked rather than assumed: no message, log record or __repr__ it produces carries one.

LISTEN and NOTIFY

conn.listen("changes")
conn.notify("changes", "payload")
conn.listening()                       # what this connection is subscribed to

for note in conn.notifications(timeout=5.0):
    print(note.channel, note.payload)

Channel names are quoted, and announcing goes through pg_notify with the name as a parameter, so a name that needs quoting behaves the same in both directions.

Prefer one style or the other, not both. psycopg changed the semantics of mixing a handler with the iterator twice and then made it a warning; pick the handler as primary and treat the iterator as the alternative.

Two-phase commit

psycopg's own methods work unchanged, including for a write that returned rows:

conn.tpc_begin(conn.xid(0, "gid", "branch"))
conn.execute_query("CREATE (:Thing)")
conn.tpc_prepare()
conn.tpc_commit()

Two things to know. The connection must not be in autocommit. And max_prepared_transactions is 0 by default, so on an untouched server tpc_prepare() raises NotSupportedError naming the setting, which is the server telling you what to change.

Generic tools and SQLAlchemy

The PEP 249 names are on agensgraph.dbapi, so anything that drives databases generically drives this: connect, apilevel, threadsafety, paramstyle, the type constructors, and the exception hierarchy re exported so DBAPIError.instance() keys off the right classes.

connection.closed and connection.broken are exposed, which is what lets a dialect decide whether a connection is dead without matching on error strings.

A whole SQLAlchemy dialect is ten lines:

from sqlalchemy.dialects.postgresql.psycopg import PGDialect_psycopg

class AgensGraphDialect(PGDialect_psycopg):
    driver = "agensgraph"

    @classmethod
    def import_dbapi(cls):
        import agensgraph.dbapi
        return agensgraph.dbapi

Version detection is free, because PostgreSQL 18beta1 (AgensGraph 2.18-devel) already matches the PostgreSQL dialect's own regex. One thing to know if you write Cypher through SQLAlchemy: (:Label) collides with SQLAlchemy's :param syntax, so use text() with bound parameters or escape the colon.

Server versions

The driver reads AgensGraph 2.17 and later, and refuses anything older at connect rather than at whichever later query first wants a catalog it does not have. It learns which it is talking to from the agversion parameter the server sends at startup, so the check costs no round trip.

The floor is what the build tests against: every release named here has the whole suite run against it on every change, so support is a thing that is checked rather than a thing that is claimed.

caps = conn.capabilities
caps.version                      # (2, 18)
caps.reported                     # '2.18-devel'
caps.has_property_promotion()     # a property stored in a column of its own
caps.has_gql_clauses()            # LET, NEXT, FINISH, FILTER, FOR, CALL
caps.has_element_ordering()       # ORDER BY on a vertex or an edge
caps.has_endpoint_elision()       # visible in a plan

Each takes check=True to raise CapabilityError naming the required version instead of returning False.

A capability read from the version describes a release, not the build in front of you. Two servers reporting 2.18-devel were found to differ, one carrying the catalog a promoted property is recorded in and one not. Where the answer has to be right rather than free, ask the server:

conn.can_promote_properties()     # asks the catalog, once per connection, and keeps the answer

The wire and type layer is identical across every supported release, even though they sit on different PostgreSQL bases (2.17 on 17, 2.18 and main on 18), so binary graph ids, the composite decode, write counters, introspection and the commit resolution path need no version gating at all. What is 2.18 only is property promotion, the GQL clause set, and ordering by a graph element.

Performance

The design decisions worth knowing, since they shape what is fast:

A property map is decoded on first touch, not on arrival. A read that never looks inside a vertex never pays for its map.
Rows are structs the collector does not track. A large result is close to invisible to it. agensgraph.paused_collection() pauses collection around a bulk read anyway, and agensgraph.freeze_after_import() moves everything alive at startup out of the collector's way.
Parsing is not the round trip. Everything above makes turning bytes into Python objects cheaper, and against a database on another machine that is a small share of what a query costs: the round trip dominates, and after it, whether the query uses an index. Measured over a couple of thousand vertices on a local server, skipping the property decode entirely is worth about 1.3 times the parse and close to nothing end to end. Expect these to show on large results and on embedding-sized maps, where the parse is the bottleneck, and to be invisible on a query returning twenty rows.
COPY for bulk, stream for large reads, Vector for embeddings. Each is an order of magnitude over the obvious alternative.

API reference

Everything exported from agensgraph. The submodules agensgraph.columnar, agensgraph.vector, agensgraph.errors and agensgraph.dbapi carry the rest.

Connecting


`connect(conninfo, **kwargs)`	open a blocking connection
`Connection` / `AsyncConnection`	the connection classes, both psycopg subclasses
`Capabilities`	what one server can do, built from the version at no round trip
`MINIMUM_VERSION`	`(2, 17)`, the oldest server this driver reads

Pools


`ConnectionPool` / `AsyncConnectionPool`	keeps connections, with generations, drain and deadlines
`NullConnectionPool` / `AsyncNullConnectionPool`	keeps none, for one request per process

Values read from the wire


`Vertex`, `Edge`, `Path`	what a graph query returns
`GraphId`	an identity, `labid` and `locid`; `LABID_MAX`, `LOCID_MAX` are their bounds
`Vector`, `SparseVector`	embeddings, lazy and sparse
`Jsonb`	psycopg's wrapper, re-exported for a value you want sent as jsonb explicitly

Values you pass in


`Label`	a label or property name to be placed into a statement as an identifier
`Unspecified`	a string sent with no type, which is psycopg's own default behaviour
`Distance`	`L2`, `INNER_PRODUCT`, `COSINE`, `L1`, `HAMMING`, `JACCARD`
`DesiredIndex`, `Unique`, `Check`, `DesiredLabel`	what you want to exist, for the reconcilers

What a statement gives back


`Result`	`records`, `keys`, `counts`, `oids`
`GraphWriteCounts`	the five counters, each `None` when it cannot be attributed
`CommitOutcome`	`COMMITTED`, `ABORTED`, `IN_PROGRESS`, `UNKNOWN`

What the catalogs give back


`Graph`	`name`, `schema`, `labels`
`LabelInfo`	`id`, `name`, `kind`, `parent`
`DeclaredProperty`	`label`, `name`, `type`, `nullable`
`Index`, `Constraint`	`label`, `name`, `unique`, `definition`
`IndexElement`	one property an index is keyed on, with its operator class and order

Reliability


`connection.ensure_labels(desired)`	make the labels asked for; never drops one
`create_index_statement`, `create_constraint_statement`, `create_label_statement`	the DDL, for a caller with no connection
`connection.describe()`	labels, properties, triples and counts, without a scan
`GraphDescription`, `Triple`, `PropertyShape`	what it returns
`connection.upsert_vertices(label, key, rows)`	copy what is missing, optionally refresh the rest
`connection.upsert_edges(label, edges)`	the same, keyed on the pair of elements each edge joins
`to_json(value)`	JSON bytes, without decoding a map nobody read
`to_builtins(value)`, `json_default`	the same shape as dicts, and for `json.dumps`
`connection.pipeline_query(statements)`	a burst of reads, every answer read back
`connection.deadline(seconds)`	bound the statements in a block, and hand back the budget
`connection.read_only_transaction()`	a transaction the server will not let write
`connection.can_run_server_programs()`	whether this role could run a command on the host
`Deadline`, `Expired`	one budget across the wait and the statement; `Expired` is a `TimeoutError`
`RetryPolicy`	decides whether and when to try again; you keep the loop
`Retryability`, `retryability(exc)`, `is_retryable(exc)`	the six categories, and how to ask
`TokenBucket`	the process-wide allowance a counter cannot multiply

Numbers


`read_numbers_exactly(bool)`	read a non-integer as a `Decimal`
`reading_numbers_exactly()`	whether that is currently on

Observability


`add_query_logger(fn)`, `remove_query_logger(fn)`	every statement, with `QueryRecord`
`enable_tracing(tracer=None)`, `disable_tracing()`	a span per statement
`Notice`	something the server said during a statement
`Notify`	an asynchronous notification, from `notifications()`

Errors


`BatchFailed`	a pipeline failed as a batch, carrying what was sent
`show_error_details(bool)`, `showing_error_details()`	whether DETAIL and CONTEXT reach the message
`redact_details(exc)`	cut them from one exception
`safe_message(exc)`	the code and the primary line, for handing somewhere else
`errors`	the whole hierarchy, plus `translate`, `mask_dsn`, `attach_query`

Bulk and memory


`paused_collection()`	pause the collector around a bulk read
`freeze_after_import()`	move what is already alive out of its way

Submodules


`columnar`	`to_arrow`, `to_pandas`, `to_polars`, `batches`, `reader`, `columns`, `Layout`; `async_to_arrow`, `async_to_pandas`, `async_to_polars`, `async_batches` for a source that is awaited
`vector`	`Vector`, `SparseVector`, `vector_index`, `nearest`, `generated_column`, `Distance`, `SEARCH_OPTIONS`
`dbapi`	the PEP 249 surface
`errors`	the exception hierarchy and the classification function

Development

uv sync --group dev --all-extras
uv run python -X dev -W error -m pytest -q          # offline; the live tests skip themselves
AGENSGRAPH_TEST_DSN="host=localhost dbname=test" uv run python -m pytest -q

The blocking interface is generated from the awaiting one:

uv run python tools/async_to_sync.py           # regenerate
uv run python tools/async_to_sync.py --check   # fail if it is out of date

Continuous integration runs the offline suite on Python 3.11 through 3.14, the full suite against AgensGraph v2.17, v2.18 and main (each with pgvector built against it), the linter and formatter, mypy --strict, a resolution at the lowest declared versions, a fuzzer over the text reader, and an install of the built wheel and source distribution into clean environments.

Many tests pin engine behaviour the driver works around, deliberately: if a later release fixes one, that test fails and names the workaround to drop.

Relationship to 1.x

2.0 is a rewrite with no backwards compatibility. 1.x was a psycopg2 type-extension shim of about nine hundred lines, and it was incorrect on output the engine legitimately produces: an empty path raised, NULL elements had no representation, a label containing { collapsed a whole path into one token, an unanchored graph id match turned 7.9.5 into (7,9), every returned value was unhashable, and len(path) made a valid single-vertex path falsy.

The distribution name and the import name are unchanged. 1.x remains on a maintenance branch for security only, and the psycopg2 line does not migrate.

License

Apache 2.0.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
src/agensgraph		src/agensgraph
tests		tests
tools		tools
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml

Uh oh!

Latest commit

History

Folders and files

Repository files navigation