General explanation of how epoll works?

2024/10/10 12:22:51

I'm doing a technical write-up on switching from a database-polling (via synchronous stored procedure call) to a message queue (via pub/sub). I'd like to be able to explain how polling a database is vastly different and much heavier than setting up a connection to a AMQP broker and configuring a message handler.

Can someone maybe provide an explanation here, or point me to a good high level tutorial on how epoll works when notifying of new data becoming available on a socket?

Answer

I assume by "how epoll works" you're referring to how it works from the user point of view (as in, how your code gets notified, and should deal with it), as opposed to the kernel point of view (as in, how epoll is implemented).

The short version is very simple: It's just like poll, except for two things:

  1. It uses a handle to an opaque data structure so you're not passing as much data back and forth across the kernel boundary.
  2. It has options that poll doesn't have (notably edge triggering and one-shot notifications) that can let you write more efficient code in certain situations.

(There's also the fact that it only works on linux. BSD and related systems have kqueue, a significantly different way to get some of the same advantages, Solaris has /dev/poll, etc., and some *nixes have nothing equivalent. So if you want to write portable code, you either have to use poll, or use some higher-level library like libevent, or write the equivalent of libevent yourself.)

If you already understand select and poll, the the Wikipedia article and the blog post linked in its References should, between them, tell you almost everything you need to know, and the man page will fill in any gaps.

If not, go learn about poll first, and only then will it make sense to learn how epoll is different.

I'm still not sure how this relates to your main question at all. You can epoll an inotify on a database file, or a pipe or socket underlying a messaging system, or just about anything else that can be represented as a file descriptor in linux, so I'm not sure how understanding epoll will help you explain the differences between polling a database vs. polling a message queue. There are of course vast differences between the two, but the event-triggering mechanism is not one of them.

https://en.xdnf.cn/q/69894.html

Related Q&A

How to return cost, grad as tuple for scipys fmin_cg function

How can I make scipys fmin_cg use one function that returns cost and gradient as a tuple? The problem with having f for cost and fprime for gradient, is that I might have to perform an operation twice…

n-gram name analysis in non-english languages (CJK, etc)

Im working on deduping a database of people. For a first pass, Im following a basic 2-step process to avoid an O(n^2) operation over the whole database, as described in the literature. First, I "b…

How to retrieve all the attributes of LDAP database

I am using ldap module of python to connect to ldap server. I am able to query the database but I dont know how to retrieve the fields present in the database, so that I can notify the user in advance …

Why would this dataset implementation run out of memory?

I follow this instruction and write the following code to create a Dataset for images(COCO2014 training set)from pathlib import Path import tensorflow as tfdef image_dataset(filepath, image_size, batch…

Paramiko: Creating a PKey from a public key string

Im trying to use the SSH protocol at a low level (i.e. I dont want to start a shell or anything, I just want to pass data). Thus, I am using Paramikos Transport class directly.Ive got the server side d…

Appending to the end of a file in a concurrent environment

What steps need to be taken to ensure that "full" lines are always correctly appended to the end of a file if multiple of the following (example) program are running concurrently.#!/usr/bin/e…

Cython Pickling in Package not found as Error

Im having trouble pickling a Cython class, but only when its defined inside a package. This problem was noted previously online, but they didnt state how it was resolved. There are two components here:…

How can I process images faster with Python?

Id trying to write a script that will detect an RGB value on the screen then click the x,y values. I know how to perform the click but I need to process the image a lot faster than my code below curren…

KFolds Cross Validation vs train_test_split

I just built my first random forest classifier today and I am trying to improve its performance. I was reading about how cross-validation is important to avoid overfitting of data and hence obtain bett…

Using Keras, how can I input an X_train of images (more than a thousand images)?

My application is accident-avoidance car systems using Machine Learning (Convolutional Neural Networks). My images are 200x100 JPG images and the output is an array of 4 elements: the car would move le…