Goodreads Developers discussion
examples / showcase
>
Python wrapper
date
newest »
newest »
Thanks for the contribution Sefa!
Thanks Sefa. I've moved your post to the examples/showcase folder and marked it as sticky, so other people can find it easily. This kind of question has come up frequently. Again, thanks so much!
Thanks for moving it to the examples/showcase folder. I must say that it is far from complete yet, but I am planning to complete it soon with proper documentation and testing.Hope it will be useful for Goodreads people!
Cheers,
Sefa wrote: "Thanks for moving it to the examples/showcase folder. I must say that it is far from complete yet, but I am planning to complete it soon with proper documentation and testing.Hope it will be usef..."
Thank you!
Hi again,I guess the python module (https://github.com/sefakilic/goodreads/) is in good shape. Any feedback/contribution is more than welcome!
Thanks!
Nice! I'll be trying this out at some point.
Hi! Junior developer here. :-)First off, thanks for writing this!
I seem to be getting an " ImportError: No module named 'session' " error on the first line, "from goodreads import client"
I'm on Windows 8.1, running Python 3.4.3. I installed the dependencies prior to installation with pip.
I'll continue to try to find a way to work around it.
Below is a log of the installation:
C:\downloads\goodreads-master>python setup.py install
running install
running bdist_egg
running egg_info
writing goodreads.egg-info\PKG-INFO
writing requirements to goodreads.egg-info\requires.txt
writing dependency_links to goodreads.egg-info\dependency_links.txt
writing top-level names to goodreads.egg-info\top_level.txt
reading manifest file 'goodreads.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'goodreads\apikey.py'
writing manifest file 'goodreads.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
creating build\bdist.win-amd64\egg
creating build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\author.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\book.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\client.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\comment.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\event.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\group.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\owned_book.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\request.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\review.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\session.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\user.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\user_status.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\__init__.py -> build\bdist.win-amd64\egg\goodreads
byte-compiling build\bdist.win-amd64\egg\goodreads\author.py to author.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\book.py to book.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\client.py to client.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\comment.py to comment.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\event.py to event.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\group.py to group.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\owned_book.py to owned_book.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\request.py to request.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\review.py to review.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\session.py to session.cpython-34.pyc
File "build\bdist.win-amd64\egg\goodreads\session.py", line 34
print self.request_token, self.request_token_secret
^
SyntaxError: Missing parentheses in call to 'print'
byte-compiling build\bdist.win-amd64\egg\goodreads\user.py to user.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\user_status.py to user_status.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\__init__.py to __init__.cpython-34.pyc
creating build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\PKG-INFO -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\SOURCES.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\dependency_links.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\requires.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\top_level.txt -> build\bdist.win-amd64\egg\EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist\goodreads-0.2.3-py3.4.egg' and adding 'build\bdist.win-amd64\egg'
to it
removing 'build\bdist.win-amd64\egg' (and everything under it)
Processing goodreads-0.2.3-py3.4.egg
Removing c:\python34\lib\site-packages\goodreads-0.2.3-py3.4.egg
Copying goodreads-0.2.3-py3.4.egg to c:\python34\lib\site-packages
goodreads 0.2.3 is already the active version in easy-install.pth
Installed c:\python34\lib\site-packages\goodreads-0.2.3-py3.4.egg
Processing dependencies for goodreads==0.2.3
Searching for rauth==0.7.1
Best match: rauth 0.7.1
Adding rauth 0.7.1 to easy-install.pth file
Using c:\python34\lib\site-packages
Searching for requests==2.6.0
Best match: requests 2.6.0
Adding requests 2.6.0 to easy-install.pth file
Using c:\python34\lib\site-packages
Searching for xmltodict==0.9.2
Best match: xmltodict 0.9.2
Adding xmltodict 0.9.2 to easy-install.pth file
Using c:\python34\lib\site-packages
Searching for nose==1.3.6
Best match: nose 1.3.6
Processing nose-1.3.6-py3.4.egg
nose 1.3.6 is already the active version in easy-install.pth
Installing nosetests-3.4-script.py script to C:\Python34\Scripts
Installing nosetests-3.4.exe script to C:\Python34\Scripts
Installing nosetests-script.py script to C:\Python34\Scripts
Installing nosetests.exe script to C:\Python34\Scripts
Using c:\python34\lib\site-packages\nose-1.3.6-py3.4.egg
Finished processing dependencies for goodreads==0.2.3
C:\Python34\Scripts>python
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from goodreads import client
Traceback (most recent call last):
File "", line 1, in
File "C:\Python34\lib\site-packages\goodreads-0.2.3-py3.4.egg\goodreads\client.py", line 2, in
from session import GoodreadsSession
ImportError: No module named 'session'
>>>
Hi, thanks for your interest.It seems you're using Python 3. The wrapper supports 2.7 only for now.
For further discussion/bug reports, you can create an issue at https://github.com/sefakilic/goodread...
Thanks!
Ooooh I see.I'll try installing Python 2.
Thank you so much!
I have written a simple Jupyter notebook to show how to use the Goodreads Python library for retrieving basic information from Goodreads:https://github.com/pirmoradian/Goodre...
Hope that helps.
Sahar
Does the Python wrapper provide the 1 request per second limitation of the terms of service or should the program using the wrapper ensure calls are not made more often than this limit?
It's been a bit since this Python wrapper was updated and is currently broken now that the Goodreads API enforces using https. I recently detached a fork of the project and changed a few things around. Please check out https://github.com/thejessleigh/bette... and let me know if you have any suggestions, or maybe open up a pull request! :)You can install the package with pip
`pip install betterreads`
Hi Jess, you rock. I'm gonna check it out!
I'm trying to use your library, but I'm getting 403 forbidden errors for any requests that require authentication. However, when I try the same request with requests_oauthlib directly it works. I'd appreciate any guidance you could provide.
Hi Michael. Which library are you using? The goodreads and goodreads2 libraries don’t use https for their authentication requests. I’ve created an updated library called betterreads that should fix the 403 problem. If you are seeing 403s using betterreads please either file a bug or direct message me and I’ll try to troubleshoot with you.
I meant to say or send me a direct message. You can view the code and file bug reports at github.com/thejessleigh/betterreads
Hi! I have not tried the installation on windows. It seems like there might be a compatibility issue with one of the dependencies on windows. I will take a look over the next couple of days. Thanks for letting me know.
How can I get all owned books of a user. By using the default listofbooks = user.owned_books() I get only five books or so.
listofbooks = user.owned_books(page=2)
causes an error message...
What am I doing wrong
using the latest better reads with python 3.7 on Mac
Owned books is a specific goodreads construct for when someone checks “I own this book” in their review. It creates a record for the condition and purchase location of the book. What exactly are you trying to get? All of the books a user has read? All of the books a user has ever shelved?
Does this wrapper support me getting all my shelves and or all books on one of my shelves?
I am looking to extract all books on all my shelves into a database
Hi Shane,You should be able to get all of the books and reviews from one shelf using this function. https://betterreads.readthedocs.io/en...
You can get all of the books associated with a user by running this function for each of their "exclusive" shelves, as each of their shelved books will belong to one of these. Let me know if you encounter any difficulties with this.
Be aware that if you have a shelf with more than a couple hundred books, it could take a minute or more to complete the operation.
As far as extracting all books and storing them in your own database, just make sure you're familiar with the developers terms of service :) https://www.goodreads.com/api/terms
Thank you so much Jess. I used to program many years ago and now do dba work. I am learning python. I used to export goodreads to csv weekly and import into my personal db. Only my data for my personal analysis. I would like to make a python program that extracts the data into my db so that I can remove the manual step.
Jess,I am new to python. I got betterreads installed and I got connected to goodreads and was able to return a single book or user name.
however, I cannot get a list of shelves and their properties returned.
I have tried various print or for loops to access the shelves but no luck.
I am trying to get a comma separated list of 1 rows per shelf with all its properties.
gid, name, exclusive, ......
sample:
from betterreads import client
gc = client.GoodreadsClient('key', 'secret')
user = gc.user(user_id=10961320)
s = user.shelves
print (s[0])
any help or sample would be much appreciated.
Shane
Hi Shane.It's been a little bit since I worked on this library, so pardon me if I ask some dumb questions.
Did you try using the Client's authenticate method to get user authentication and access their data? Some information in the Goodreads API is gated behind the user agreeing to allow your third party app to access it. I definitely need to update the documentation to reflect this.
Take a look at this section in the README and let me know if you need additional help. Also happy to chat via DM or via Github issues. Github is a bit easier since it supports code formatting.
https://github.com/thejessleigh/bette...
Jess,(posted in github too)
I apologize if I am just missing something as a new python user.
Your documentation for user:
https://betterreads.readthedocs.io/en...
I can user example and get user.user and user.user_name to work however nothing below your sample code works.
list_groups, owned_books, shelves, etc do not work.
here is test commands( name, link work, but shelves and list_groups do not):
>>> from betterreads import client
>>> gc = client.GoodreadsClient('key', 'secret key')
>>> user = gc.user(user_id=10961320)
>>> user.name
'Shane Phillips'
>>> user.shelves
Traceback (most recent call last):
File "", line 1, in
TypeError: __repr__ returned non-string (type int)
>>> user.link
'https://www.goodreads.com/user/show/1...'
>>> user.list_groups
Traceback (most recent call last):
File "", line 1, in
TypeError: __repr__ returned non-string (type int)
Hey Shane - I just looked at a recent pull request on the project and it looks as though some integration tests that were previously passing are now failing. I wonder if that has something to do with the issues you are seeing. It's possible something has changed in the API that I haven't accounted for. I'm very sorry - I've been only moderately available due to the holiday week last week. Let me poke around and try to figure out what's going on.
Jess wrote: "... something has changed in the API..."I pulled manually via json and they changed the API indeed. Sadly without notice (?)
thanks Sven. do you know what change? all methods or specific ones?
Hi Shane (and all) - can answer not very precise, because I'm just starting dev. I'm on python, learning with and reusing code from qs_ledger (quantified self python tool).It seems that a query on /user/show/ gave back a reviews_count, now that query breaks. Haven't looked into it before it broke, so can't tell what's the different return value now and then.
> user_info_url = get_user_info_url(api_key, user_id)
>user_info_data = urllib.request.urlopen(user_info_url).read()
>user_info_dict = xmltodict.parse(user_info_data)
>books_total = int(user_info_dict["GoodreadsResponse"]["user"]["reviews_count"]["#text"])
>user_name = user_info_dict["GoodreadsResponse"]["user"]["user_name"]
>user_info = {
> "user_name" : user_name,
> "books_total" : books_total
> }
>return(user_info)
Then I tried rewrite my code using the wrapper above, but to no success.
So I believe the API changed between 29/12/2019 (last correct query) and 2/2/2020.
I am using "ElementTree" to extract the Goodreads API. However, I cannot find out how to get my list of shelves that I have a book on. note: added a space so goodreads would not process xml.
-------sample----------
< spoilers_state>none
< shelves>
< shelf exclusive="false" id="153934772" name="full-list" review_shelf_id="2562838816" sortable="true"/>
< shelf exclusive="true" id="269364260" name="s-harry_bosch" review_shelf_id="2562838881" sortable="true"/>
< /shelves>
< recommended_for/>
---------end sample
I am not sure how to get the shelf name and shelf_id out of these tags.
any help?
API method:
https://www.goodreads.com/review/list...
thanks
I apologize for my lack of response on this. This is a side project that I just have not had the bandwidth to keep up to date. I am open to looking at PRs and I hope that I will have some time in the late winter/early spring to release an updated version that accounts for some of the reported errors.
Jess, no hurry! Enjoy Winter and thanks for your work.
my "review_counts" problem was simply the privacy settings on my profile page. odd, as I'm authenticated, but ok, it works again.
Jess wrote: "I apologize for my lack of response on this. This is a side project that I just have not had the bandwidth to keep up to date. I am open to looking at PRs and I hope that I will have some time in t..."No worries.
Anybody who's using python right now to authenticate, fetch user's book list, user ratings, average ratings, etc can check out my implementation. I've used a combo of the Goodreads python wrapper by Sefa and get requests to the Goodreads API itself.https://github.com/suraj-ranganath/bookpal
What is the best way right now to access the Goodreads API through Python? Would I need to look into @Suraj's solution or is there something that is already a bit more standardized so that I can incorporate it directly. The goal is eventually to build a Discord-bot that can call-up book info for our next bookclub.
I use elementtree library. I wrote program (my first python) to download all my books to a sqlite db. # Shane Phillips, 2019
# load goodreads.com book data to database via API
import requests, sqlite3, time, sys, getopt
from xml.etree import ElementTree
import os.path
from os import path
from datetime import datetime
I would be glad to share it. Nothing special about it. message me directly and I can email to you.
Thanks, I'll take any help I can get, because I am rather new to this and getting a few early pointers do make the difference!
I've started a Python wrapper for Goodreads API. It aims to provide full coverage of Python interface for API methods. Any suggestions or contributions are welcomed!
https://github.com/sefakilic/goodreads