Contents | Previous (3.2 More on Functions) | Next (3.4 Modules)

3.3 Error Checking

Although exceptions were introduced earlier, this section fills in some additional details about error checking and exception handling.

How programs fail

Python performs no checking or validation of function argument types or values. A function will work on any data that is compatible with the statements in the function.

defadd(x, y):
returnx+yadd(3, 4) # 7add('Hello', 'World') # 'HelloWorld'add('3', '4') # '34'

If there are errors in a function, they appear at run time (as an exception).

defadd(x, y):
returnx+y>>>add(3, '4')
Traceback (mostrecentcalllast):
...
TypeError: unsupportedoperand type(s) for+:
'int'and'str'>>>

To verify code, there is a strong emphasis on testing (covered later).

Exceptions

Exceptions are used to signal errors. To raise an exception yourself, use raise statement.

ifnamenotinauthorized:
raiseRuntimeError(f'{name} not authorized')

To catch an exception use try-except.

try:
authenticate(username)
exceptRuntimeErrorase:
print(e)

Exception Handling

Exceptions propagate to the first matching except.

defgrok():
...
raiseRuntimeError('Whoa!') # Exception raised heredefspam():
grok() # Call that will raise exceptiondefbar():
try:
spam()
exceptRuntimeErrorase: # Exception caught here
...
deffoo():
try:
bar()
exceptRuntimeErrorase: # Exception does NOT arrive here
...
foo()

To handle the exception, put statements in the except block. You can add any statements you want to handle the error.

defgrok(): ...
raiseRuntimeError('Whoa!')
defbar():
try:
grok()
exceptRuntimeErrorase: # Exception caught herestatements# Use this statementsstatements
...
bar()

After handling, execution resumes with the first statement after the try-except.

defgrok(): ...
raiseRuntimeError('Whoa!')
defbar():
try:
grok()
exceptRuntimeErrorase: # Exception caught herestatementsstatements
...
statements# Resumes execution herestatements# And continues here
...
bar()

Built-in Exceptions

There are about two-dozen built-in exceptions. Usually the name of the exception is indicative of what's wrong (e.g., a ValueError is raised because you supplied a bad value). This is not an exhaustive list. Check the documentation for more.

ArithmeticErrorAssertionErrorEnvironmentErrorEOFErrorImportErrorIndexErrorKeyboardInterruptKeyErrorMemoryErrorNameErrorReferenceErrorRuntimeErrorSyntaxErrorSystemErrorTypeErrorValueError

Exception Values

Exceptions have an associated value. It contains more specific information about what's wrong.

raiseRuntimeError('Invalid user name')

This value is part of the exception instance that's placed in the variable supplied to except.

try:
...
exceptRuntimeErrorase: # `e` holds the exception raised
...

e is an instance of the exception type. However, it often looks like a string when printed.

exceptRuntimeErrorase:
print('Failed : Reason', e)

Catching Multiple Errors

You can catch different kinds of exceptions using multiple except blocks.

try:
...
exceptLookupErrorase:
...
exceptRuntimeErrorase:
...
exceptIOErrorase:
...
exceptKeyboardInterruptase:
...

Alternatively, if the statements to handle them is the same, you can group them:

try:
...
except (IOError,LookupError,RuntimeError) ase:
...

Catching All Errors

To catch any exception, use Exception like this:

try:
...
exceptException: # DANGER. See belowprint('An error occurred')

In general, writing code like that is a bad idea because you'll have no idea why it failed.

Wrong Way to Catch Errors

Here is the wrong way to use exceptions.

try:
go_do_something()
exceptException:
print('Computer says no')

This catches all possible errors and it may make it impossible to debug when the code is failing for some reason you didn't expect at all (e.g. uninstalled Python module, etc.).

Somewhat Better Approach

If you're going to catch all errors, this is a more sane approach.

try:
go_do_something()
exceptExceptionase:
print('Computer says no. Reason :', e)

It reports a specific reason for failure. It is almost always a good idea to have some mechanism for viewing/reporting errors when you write code that catches all possible exceptions.

In general though, it's better to catch the error as narrowly as is reasonable. Only catch the errors you can actually handle. Let other errors pass by--maybe some other code can handle them.

Reraising an Exception

Use raise to propagate a caught error.

try:
go_do_something()
exceptExceptionase:
print('Computer says no. Reason :', e)
raise

This allows you to take action (e.g. logging) and pass the error on to the caller.

Exception Best Practices

Don't catch exceptions. Fail fast and loud. If it's important, someone else will take care of the problem. Only catch an exception if you are that someone. That is, only catch errors where you can recover and sanely keep going.

`finally` statement

It specifies code that must run regardless of whether or not an exception occurs.

lock=Lock()
...
lock.acquire()
try:
...
finally:
lock.release() # this will ALWAYS be executed. With and without exception.

Commonly used to safely manage resources (especially locks, files, etc.).

`with` statement

In modern code, try-finally is often replaced with the with statement.

lock=Lock()
withlock:
# lock acquired
...
# lock released

A more familiar example:

withopen(filename) asf:
# Use the file
...
# File closed

with defines a usage context for a resource. When execution leaves that context, resources are released. with only works with certain objects that have been specifically programmed to support it.

Exercises

Exercise 3.8: Raising exceptions

The parse_csv() function you wrote in the last section allows user-specified columns to be selected, but that only works if the input data file has column headers.

Modify the code so that an exception gets raised if both the select and has_headers=False arguments are passed. For example:

>>>parse_csv('Data/prices.csv', select=['name','price'], has_headers=False)
Traceback (mostrecentcalllast):
File"<stdin>", line1, in<module>File"fileparse.py", line9, inparse_csvraiseRuntimeError("select argument requires column headers")
RuntimeError: selectargumentrequirescolumnheaders>>>

Having added this one check, you might ask if you should be performing other kinds of sanity checks in the function. For example, should you check that the filename is a string, that types is a list, or anything of that nature?

As a general rule, it’s usually best to skip such tests and to just let the program fail on bad inputs. The traceback message will point at the source of the problem and can assist in debugging.

The main reason for adding the above check to avoid running the code in a non-sensical mode (e.g., using a feature that requires column headers, but simultaneously specifying that there are no headers).

This indicates a programming error on the part of the calling code. Checking for cases that "aren't supposed to happen" is often a good idea.

Exercise 3.9: Catching exceptions

The parse_csv() function you wrote is used to process the entire contents of a file. However, in the real-world, it’s possible that input files might have corrupted, missing, or dirty data. Try this experiment:

>>>portfolio=parse_csv('Data/missing.csv', types=[str, int, float])
Traceback (mostrecentcalllast):
File"<stdin>", line1, in<module>File"fileparse.py", line36, inparse_csvrow= [func(val) forfunc, valinzip(types, row)]
ValueError: invalidliteralforint() withbase10: ''>>>

Modify the parse_csv() function to catch all ValueError exceptions generated during record creation and print a warning message for rows that can’t be converted.

The message should include the row number and information about the reason why it failed. To test your function, try reading the file Data/missing.csv above. For example:

>>>portfolio=parse_csv('Data/missing.csv', types=[str, int, float])
Row4: Couldn't convert ['MSFT', '', '51.23']
Row4: Reasoninvalidliteralforint() withbase10: ''Row7: Couldn't convert ['IBM', '', '70.44']
Row7: Reasoninvalidliteralforint() withbase10: ''>>>>>>portfolio
[{'price': 32.2, 'name': 'AA', 'shares': 100}, {'price': 91.1, 'name': 'IBM', 'shares': 50}, {'price': 83.44, 'name': 'CAT', 'shares': 150}, {'price': 40.37, 'name': 'GE', 'shares': 95}, {'price': 65.1, 'name': 'MSFT', 'shares': 50}]
>>>

Exercise 3.10: Silencing Errors

Modify the parse_csv() function so that parsing error messages can be silenced if explicitly desired by the user. For example:

>>>portfolio=parse_csv('Data/missing.csv', types=[str,int,float], silence_errors=True)
>>>portfolio
[{'price': 32.2, 'name': 'AA', 'shares': 100}, {'price': 91.1, 'name': 'IBM', 'shares': 50}, {'price': 83.44, 'name': 'CAT', 'shares': 150}, {'price': 40.37, 'name': 'GE', 'shares': 95}, {'price': 65.1, 'name': 'MSFT', 'shares': 50}]
>>>

Error handling is one of the most difficult things to get right in most programs. As a general rule, you shouldn’t silently ignore errors. Instead, it’s better to report problems and to give the user an option to the silence the error message if they choose to do so.