Non-blocking IO

[oduortoni]

Goal: Non-blocking IO

Current Flow:

• main.c → calls http.ListenAndServe()

• http_server.c → calls net_serve()

• net_serve.c → accepts connections in a blocking loop

• Each connection calls handle_protocol() synchronously

Dependencies: possibly the template system (*)

[koutoftimer]

I'd say non-blocking IO and libuv. Non-blocking IO for web servers as well as libuv is almost the standard nowadays.

[koutoftimer]

Also, it feels like over engineering at this point. C is capable of doing some job even in single thread (I've seen person implementing production game server that way, and the game was quite popular at that time). IMHO, it is better to focus an effort on architecture and its implementation because right now it all, basically, bare bones.

[oduortoni]

I was actually aware about the fact that the http layer is not fully free from the lower sockets layer. I did not have a concrete plan at the time. But now that you have mentioned libuv, it gives me ideas from the Node-js and GO world. How about, instead of using:

write(client.socket, ...)

we introduce an interface that has a write and read. That way, we can actually mock network calls when testing different things. Also, this way, we have a http layer that imposes a requirement about the underlying data provider, be it a socket or a file, as long as it has a way to read or write data from it. I don't know whether you know Golang, but I borrow my ideas from all over the place and GO just does this perfectly. It has interfaces everywhere that make it easy to separate layers.

Beyond that, I have no idea how to make the separation right now. I could use a bit of time to come up with a clear separation; not just for this, but other things. You can also come up with other ideas.

[koutoftimer]

I don't know whether you know Golang

Not too good. I've implemented like a 100 lines HTTP server that serves static files as if they where on NTFS (case insensitively).

During that process, I've read some parts of the server implementation and saw the patterns you've introduced in this project. I like an idea of most of them but stuff like "handle this, handle that" still gives me a head ache. I'd like to came up with less opaque naming

[oduortoni]

I am actually considering the fact that our main entrypoint should not be the place to place the routes. I think a proper place would be within the app directory where the user can edit a routes.c file. We could borrow the infamous router file and have:

http.route("GET /", Index);
http.route("POST /", Consume);

That will actually be better because the app directory will contain everything a user needs. That is why I placed the templates directory in there.

[koutoftimer]

I'd like to borrow concept of reusable apps, which means that routes should be composable with respect to the parent app. E.g.:

assign_route(.app = main_app,    .route = "/"                             );
assign_route(.app = login_app,   .route = "/login", .parent = main_app    );

assign_route(.app = admin_panel, .route = "/admin"                        );
assign_route(.app = login_app,   .route = "/login", .parent = admin_panel );

(this is just an idea showcase, not an actual code)

[koutoftimer]

The more I think about reusable apps concept the more I realize that we can't be like Django (batteries included) and Flask, where you have to deliberately pick each piece of functionality you'll need or implement it yourself from scratch, at the same time.

[oduortoni]

So we do a bare-bones system. Anyone else can extend it later. Django is always coming up with versions due to the complexities. You help the user too much, you stand a chance of being in development every day instead of stabilizing what is present.

[koutoftimer]

You help the user too much, you stand a chance of being in development every day instead of stabilizing what is present.

No, it is not the root of the problem. The root of evil that causes unability to stabilize stuff is in the requirements. Every single piece of software has its own unique set of requirements. Each component, software consist of, has its own set of requirements. And, the more components you have, the more places where you have mismatched requirements occur. Because your requirements doesn't not match, with what is already present, you are forces to write what is needed for your requirements yourself. It has nothing to do with big or small in term of number of requirements for the specific system. Only number of mismatched requirements matter. Complexity of Django comes from the will to cover as much requirements as possible (theoretically all of them).

That is why I value flexibility and simplicity, because every single line of code brings restriction to what you can possibly do with the software and only when there is nothing - you can do everything.

[oduortoni]

Instead of apps, can we just have Routers that can include other routers. That way, you can build an entire system with its own routes independent of another then combine them under one router.

[koutoftimer]

These are the details. I just wanted to outline the scope of interest.

[koutoftimer]

There are good examples for io_uring usage via liburing that claims to simplify io_using API https://github.com/axboe/liburing/tree/master

[oduortoni]

There are good examples for io_uring usage via liburing that claims to simplify io_using API https://github.com/axboe/liburing/tree/master

Did you see their test folder? Simple assert and if statements. I like it already.

[koutoftimer]

Did you see their test folder? Simple assert and if statements. I like it already.

Do you want to get rid of gtest as a dependency?