Histogram of sum instead of count using numpy and matplotlib

2024/12/9 22:08:17

I have some data with two columns per row. In my case job submission time and area.

I have used matplotlib's hist function to produce a graph with time binned by day on the x axis, and count per day on the y axis:

import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import datetime as dtdef timestamp_to_mpl(timestamp):return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp))nci_file_name = 'out/nci.csv'
jobs = np.genfromtxt(nci_file_name, dtype=int, delimiter=',', names=True, usecols(1,2,3,4,5))fig, ax = plt.subplots(2, 1, sharex=True)
vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl)
qtime = vect_timestamp_to_mpl(jobs['queued_time'])
start_date = dt.datetime(2013, 1, 1)
end_date = dt.datetime(2013, 4, 1)
bins = mpl.dates.drange(start_date, end_date, dt.timedelta(days=1))
ax[0].hist(qtime[jobs['charge_rate']==1], bins=bins, label='Normal', color='b')
ax[1].hist(qtime[jobs['charge_rate']==3], bins=bins, label='Express', color='g')
ax[0].grid(True)
ax[1].grid(True)
fig.suptitle('NCI Workload Submission Daily Rate')
ax[0].set_title('Normal Queue')
ax[1].set_title('Express Queue')
ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator())
ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator()))
ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date))
plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right')
ax[1].set_xlabel('Date')
ax[0].set_ylabel('Jobs per Day')
ax[1].set_ylabel('Jobs per Day')
fig.savefig('out/figs/nci_sub_rate_day_sub.png')
plt.show()

NCI Workload Submission Daily Rate

I now want a graph with time binned by day on the x axis and the summed by bin area on the y axis.

So far I have come up with this using a list comprehension:

import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import datetime as dtdef timestamp_to_mpl(timestamp):return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp))def binsum(bin_by, sum_by, bins):bin_index = np.digitize(bin_by, bins)sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))]return sumsfig, ax = plt.subplots(2, 1, sharex=True)
vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl)
qtime = vect_timestamp_to_mpl(jobs['queued_time'])
area = jobs['run_time'] * jobs['req_procs']
start_date = dt.datetime(2013, 1, 1)
end_date = dt.datetime(2013, 4, 1)
delta = dt.timedelta(days=1)
bins = mpl.dates.drange(start_date, end_date, delta)
sums_norm = binsum(qtime[jobs['charge_rate']==1], area[jobs['charge_rate']==1], bins)
sums_expr = binsum(qtime[jobs['charge_rate']==3], area[jobs['charge_rate']==3], bins)
ax[0].bar(bins, sums_norm, width=1.0, label='Normal', color='b')
ax[1].bar(bins, sums_expr, width=1.0, label='Express', color='g')
ax[0].grid(True)
ax[1].grid(True)
fig.suptitle('NCI Workload Area Daily Rate')
ax[0].set_title('Normal Queue')
ax[1].set_title('Express Queue')
ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator())
ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator()))
ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date))
plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right')
ax[1].set_xlabel('Date')
ax[0].set_ylabel('Area per Day')
ax[1].set_ylabel('Area per Day')
fig.savefig('out/figs/nci_area_day_sub.png')
plt.show()

NCI Workload Area Daily Rate

I am still new to NumPy and would like to know if I can improve:

def binsum(bin_by, sum_by, bins):bin_index = np.digitize(bin_by, bins)sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))]return sums

So it doesn't use Python lists.

Is it possible to somehow explode out sum_by[bin_index==i] so I get an array of arrays, with length len(bins)? Then np.sum() would return a numpy array.

Answer

Both Matplotlib's hist function and NumPy's histogram function have a weights optional keyword argument. I think the only relevant lines to change in your first code should end up looking like:

ax[0].hist(qtime[jobs['charge_rate']==1], weights=area[jobs['charge_rate']==1],bins=bins, label='Normal', color='b')
ax[1].hist(qtime[jobs['charge_rate']==3], weights=area[jobs['charge_rate']==3],bins=bins, label='Express', color='g')
https://en.xdnf.cn/q/73168.html

Related Q&A

Find subsequences of strings within strings

I want to make a function which checks a string for occurrences of other strings within them. However, the sub-strings which are being checked may be interrupted within the main string by other letters…

How to bestow string-ness on my class?

I want a string with one additional attribute, lets say whether to print it in red or green.Subclassing(str) does not work, as it is immutable. I see the value, but it can be annoying.Can multiple inhe…

How to pass Python instance to C++ via Python/C API

Im extending my library with Python (2.7) by wrapping interfaces with SWIG 2.0, and have a graph object in which I want to create a visitor. In C++, the interface looks like this:struct Visitor{virtua…

REST API in Python with FastAPI and pydantic: read-only property in model

Assume a REST API which defines a POST method on a resource /foos to create a new Foo. When creating a Foo the name of the Foo is an input parameter (present in the request body). When the server creat…

a class with all static methods [closed]

Closed. This question is opinion-based. It is not currently accepting answers.Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.Clo…

How can I find null values with SELECT query in psycopg?

I am using psycopg2 library in python and the INSERT query works good when I insert null Value with None, but when I want to do SELECT null values, with None doesnt return any.cur.execute("SELECT …

Pause and continue stopwatch

I am trying to create stopwatch. I have done it but I would like to pause and continue the time whenever I want. I have tried some things but I have no idea how to do it. Is there anybody who would exp…

How do I escape `@` letter from SQL password in connection URI [duplicate]

This question already has an answer here:handle @ in mongodb connection string(1 answer)Closed 9 years ago.when you connect to mongodb using python from SQLAlchamey, we use mongodb://username:password@…

Set WTForms submit button to icon

I want a submit button that displays an icon rather than text. The button is a field in a WTForms form. I am using Bootstrap and Open Iconic for styling and icons. How do I set the submit field to d…

what is the significance of `__repr__` function over normal function [duplicate]

This question already has answers here:Purpose of __repr__ method?(6 answers)Closed 5 years ago.I am trying to learn python with my own and i stucked at __repr__ function. Though i have read lots of p…