"Gotta" Catch'em All exceptions
So, I was recently doing a webscraping project. I had a database of people and I had to lookup and confirm their identity. Identity in the sense that they had either studied or taught in my college. The task can be done manually for a small set, but I had 7000 records!!
So I used xgoogle for this purpose. It is a Python Library for Google Search.
Now, after having completed the task, I can name a minimum 7 exceptions that can occur when you automate web lookups. They are :
SearchError (class in xgoogle that handles exceptions raised by google during searches)
HTTPError (generally raised when there is some error in authentication, like trying to access a secured page of an organization, or when the url is http when it really should be https)
URLError (being a subclass of OSError, this error is related to system related problems like input/output . This includes error in fetching data from a url)
SocketError (Raised when there is some issue with the server eg. overload, throttle)
ValueError (Raised when the required datatype conversion is not permitted)
IncompleteRead (When the server closes the connection before the chunk of data has benn fully retireved.)
BadStatusLine (Raised when we request a webpage, and the server responds with a code that Python cannot understand)
Well this is not the complete list.
I had to automate the task in such a way that I start the code before going to bed and I get it completed when I wake up. So, I decided to catch all :
try:
my code
except KeyBoardInterrupt:
raise
except:
#handles all exceptions except KeyBoardInterrupt
This piece of code will ensure that unless you manually try to end your code, it will not stop.