Slow extend() for EventedList and friends

[ndxmrb]

Hi,
I want to insert a batch of objects into a SelectableEventedList, but I want to fire corresponding signals only once, not for each object. The purpose is to reduce update calls in widgets down the line.

I thought I'd use paused() with a reducer to collect all indexes. But inserting currently only takes an index: int, not a tuple.

Question: How can I defer signal emission properly, such that Qt models update properly?

I tried blocking, but then you need to decide what to emit when done.

Or should I rather try to throttle the model functions?

Suggestion: inserting etc. could accept tuples as well, such that Qt's beginInsertRows() and endInsertRows() can be used to reduce update calls. extend() could then make use of this directly, such that it only emits the inserted signal once.

Thanks in advance,
ndxmrb

[tlambert03]

would this work for your case?

from __future__ import annotations

from collections.abc import Sequence
from qtpy.QtCore import QAbstractItemModel, QModelIndex
from psygnal.containers import EventedList


def batch_extend(model: QAbstractItemModel, lst: EventedList, items: Sequence) -> None:
    start = len(lst)
    model.beginInsertRows(QModelIndex(), start, start + len(items) - 1)
    try:
        with lst.events.inserting.blocked(), lst.events.inserted.blocked():
            lst.extend(items)
    finally:
        model.endInsertRows()

[ndxmrb]

Hey,
thank you for the quick answer.

Yes, in principle that would work for one specific case. But it needed to be done for every model in the application and isn't a solution for widgets relying on the signals emitted from the EventedList directly.

Should such widgets throttle their update function? If so, how can the potential thread mismatch be handled, because UI elements can only be modified in the main thread?

[tlambert03]

can we use some more concrete MRE examples? I think this mostly boils down to an application question, which I'm happy to weigh in on, but I'm not sure there's a global answer here. Some of it relates to idiosyncrasies of Qt specifically (which again, we do want to be able to "have an answer for", but backing a specific QAbstractModel isn't the sole mission of psygnal)

it needed to be done for every model in the application and isn't a solution for widgets relying on the signals emitted from the EventedList directly.

one of the key issues here are the specifics of QAbstractItemModel and the beginInsertRows/endInsertRows pattern. That's a Qt-specific API... not a psygnal one. The pattern above shows you how you can drive it, and if you want to have many different QAbstractItemModel models, then you could always may a subclass that specifically takes an instance of EventedList, and give it a method that drives the extend in a way that is efficient for Qt.

And here too, I would note that you don't have to block lst.events.inserting: you could also just temporarily tell the QModel to not pay attention to it. In other words:

# not tested
class QEventedList(QAbstractItemModel):
    def batch_extend(self, lst: EventedList, items: Sequence) -> None:
        start = len(lst)
        self.beginInsertRows(QModelIndex(), start, start + len(items) - 1)
        try:
            # if you have previously connected lst.events to anything about self, disconnect here
            lst.extend(items)
        finally:
            self.endInsertRows()
            # reconnect here, if desired

Should such widgets throttle their update function?

I'm not sure why/where throttling is coming in here. so we'll need more concrete examples of the objects in play, and the thing you're trying to avoid by throttling.

how can the potential thread mismatch be handled, because UI elements can only be modified in the main thread?

Again, depends on the details. Qt connections (auto connection by default) is pretty good at marshalling signals across threads, so it really depends on what is being connected to what here. Are they both Qobjects? Only one side? etc...

---

can you give me a few more constraints on the shape of the problem you're trying to solve, and what specific things about the solution are unsatisfactory at the moment?

[ndxmrb]

Hey there,
thanks for helping!

I know psygnal isn't about backing Qt in everything what it does. Actually, I don't want to muck around with Qt models at all if possible. And I agree, it's probably an application question, and this issue is just me "holding it wrong" 😛

---

My example app would be showing a table of (x,y) coordinates using a QTableView. These coordinates should also be shown in a matplotlib plot. The ground truth data is an EventedList (ideally with selection, but let's not go there yet). The QTableView is fed via an intermediate QAbstractModel. The plot class listens to the list signals directly.

The problem I encounter is, that upon extending the list by n elements, I get n inserted signals. Therefore, the plot updates n times, when that could happen once only. (Here, throttling the update function could help?). In the table, this also happens. Qt has the beginInsertRows stuff that mitigates multiple updates for the index range I believe, but I can't use it.

---

Your fix looks to me like I can't just connect my intermediate QAbstractModel to the list signals and automate dealing with all of what should happen. I'd need to know beforehand, that multiple rows will be inserted and the model method needs to be called for those seperately. It doesn't rely on signals sent. Did I get that right?