[Eril-l] Update on COUNTER 5.1 Harvester (my free client)

[Electronic Resources in Libraries discussion list]

Hi, all.
This regards the free COUNTER 5.1 usage data harvester<https://github.com/Melissa-Belvadi-Library-Projects/C5.1_Harvester/tree/af55e9ab45a53c8929f608634097e1b23dcbfe72> that I released last fall. If your library can't afford a commercial COUNTER harvester like Celus, you will probably find this a huge time-saver.

The rest of this email is for people who did start using it before mid-December. There's no "registration" so I have no idea how to reach people who downloaded it.

In mid-December, I found a bug that affects the sqlite database when a particular major vendor fails to include some required data.
I only found the bug after the vendor started sending the bad data (which they should not be as it breaks COUNTER rules).

It does not affect the .tsv files at all.

If you did start to use the harvester including the sqlite database, I strongly recommend you delete the existing counterdata.db file (or move/rename it), and after copying the changed files into the "src" folder, re-run all of your providers for whatever entire time period you had done before (many go back before 2025).
One nice aspect of it being not compiled is that you can just directly replace individual .py files in the src folder and re-run the harvester the usual way and that's all that is needed to "update" to the latest version.

The list of individual files that were changed, which you can download individually, is:

  *
convert_counter_json_to_tsv.py
  *
fetch_json.py
  *
getcounter.py
  *
process_item_details.py

Or you could download the entire release 2.1.1 and after unzipping it, copy all of the files ending .py into your existing ".../src/ folder.

Please let me know if you have any questions about this update, or the harvester more generally, or even about COUNTER more generally.


Melissa Belvadi
Collections Librarian
University of Prince Edward Island
mbelvadi at upei.ca<mailto:mbelvadi at upei.ca>  902-566-0581
ORCID iD: 0000-0002-4433-0189
my public calendar<https://outlook.office365.com/owa/calendar/0fbab27c909e4493be65313bd66d66b6@upei.ca/5fa60af92c6d451c9ddf90c0bb11e00f15552192987609852692/calendar.html>
My pronouns are ಅವರು/ಅವರನ್ನು
My emails are sent during the hours that I work and I understand that you will respond during the hours that you work.

Make an appointment: Use YouCanBookMe https://mbelvadi.youcanbook.me/
or for other MS365 / Outlook users, including UPEI people:
[cid:092adccd-ce12-4bdb-93f3-948289619e94]<https://outlook.office.com/bookwithme/user/0fbab27c909e4493be65313bd66d66b6@upei.ca?anonymous&ismsaljsauthenabled&ep=bwmEmailSignature>
Book time to meet with me<https://outlook.office.com/bookwithme/user/0fbab27c909e4493be65313bd66d66b6@upei.ca?anonymous&ismsaljsauthenabled&ep=bwmEmailSignature>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.eril-l.org/pipermail/eril-l-eril-l.org/attachments/20260112/afbc61ff/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-rvlygpao.png
Type: image/png
Size: 528 bytes
Desc: Outlook-rvlygpao.png
URL: <http://lists.eril-l.org/pipermail/eril-l-eril-l.org/attachments/20260112/afbc61ff/attachment.png>