Read Files From Data Lake Using Python
Welcome
Hi! If yous desire to acquire how to work with files in Python, then this commodity is for you. Working with files is an of import skill that every Python developer should learn, then let's get started.
In this article, you will learn:
- How to open a file.
- How to read a file.
- How to create a file.
- How to change a file.
- How to shut a file.
- How to open files for multiple operations.
- How to piece of work with file object methods.
- How to delete files.
- How to work with context managers and why they are useful.
- How to handle exceptions that could be raised when yous piece of work with files.
- and more!
Allow'due south begin! ✨
🔹 Working with Files: Basic Syntax
One of the most important functions that you will need to use equally you work with files in Python is open() , a built-in function that opens a file and allows your program to apply it and work with information technology.
This is the basic syntax:
💡 Tip: These are the two most unremarkably used arguments to call this function. There are vi additional optional arguments. To learn more nearly them, please read this article in the documentation.
First Parameter: File
The offset parameter of the open() function is file , the absolute or relative path to the file that you lot are trying to work with.
Nosotros usually use a relative path, which indicates where the file is located relative to the location of the script (Python file) that is calling the open() function.
For example, the path in this function call:
open("names.txt") # The relative path is "names.txt" Only contains the name of the file. This can be used when the file that y'all are trying to open is in the same directory or binder every bit the Python script, like this:
But if the file is within a nested binder, similar this:
And so we demand to utilize a specific path to tell the function that the file is within some other folder.
In this example, this would be the path:
open("data/names.txt") Notice that nosotros are writing data/ first (the name of the folder followed by a /) and then names.txt (the name of the file with the extension).
💡 Tip: The 3 letters .txt that follow the dot in names.txt is the "extension" of the file, or its type. In this case, .txt indicates that it's a text file.
Second Parameter: Mode
The 2d parameter of the open up() office is the manner , a string with one character. That unmarried character basically tells Python what you are planning to do with the file in your plan.
Modes bachelor are:
- Read (
"r"). - Append (
"a") - Write (
"westward") - Create (
"x")
You lot can likewise cull to open up the file in:
- Text mode (
"t") - Binary manner (
"b")
To apply text or binary mode, y'all would need to add these characters to the master fashion. For example: "wb" ways writing in binary way.
💡 Tip: The default modes are read ("r") and text ("t"), which means "open up for reading text" ("rt"), so you don't need to specify them in open() if you lot want to employ them because they are assigned by default. Yous can simply write open(<file>).
Why Modes?
It really makes sense for Python to grant only certain permissions based what you are planning to do with the file, right? Why should Python allow your program to do more than necessary? This is basically why modes exist.
Retrieve nigh it — allowing a program to do more than than necessary tin problematic. For example, if you lot only demand to read the content of a file, it tin be unsafe to allow your program to modify it unexpectedly, which could potentially introduce bugs.
🔸 How to Read a File
Now that you know more about the arguments that the open up() function takes, allow's run into how you can open up a file and store it in a variable to employ it in your program.
This is the basic syntax:
Nosotros are merely assigning the value returned to a variable. For example:
names_file = open("data/names.txt", "r") I know y'all might exist asking: what type of value is returned by open() ?
Well, a file object.
Let's talk a little bit most them.
File Objects
According to the Python Documentation, a file object is:
An object exposing a file-oriented API (with methods such every bit read() or write()) to an underlying resources.
This is basically telling u.s. that a file object is an object that lets us work and interact with existing files in our Python program.
File objects have attributes, such as:
- name: the proper name of the file.
- closed:
Trueif the file is closed.Falseotherwise. - mode: the style used to open the file.
For case:
f = open up("data/names.txt", "a") print(f.mode) # Output: "a" Now let's encounter how you can access the content of a file through a file object.
Methods to Read a File
For us to exist able to piece of work file objects, we need to accept a manner to "collaborate" with them in our program and that is exactly what methods practise. Let's see some of them.
Read()
The starting time method that yous need to learn virtually is read() , which returns the unabridged content of the file as a string.
Here we take an instance:
f = open up("information/names.txt") print(f.read()) The output is:
Nora Gino Timmy William Y'all can use the type() part to confirm that the value returned by f.read() is a string:
print(type(f.read())) # Output <class 'str'> Aye, it'due south a string!
In this case, the entire file was printed because nosotros did not specify a maximum number of bytes, but nosotros tin can do this also.
Hither nosotros have an example:
f = open("data/names.txt") impress(f.read(3)) The value returned is limited to this number of bytes:
Nor ❗️Important: You need to close a file after the task has been completed to free the resources associated to the file. To practice this, you demand to call the close() method, similar this:
Readline() vs. Readlines()
You tin can read a file line by line with these two methods. They are slightly dissimilar, so allow's see them in item.
readline() reads one line of the file until it reaches the end of that line. A trailing newline character (\due north) is kept in the cord.
💡 Tip: Optionally, you can pass the size, the maximum number of characters that you want to include in the resulting cord.
For example:
f = open("data/names.txt") impress(f.readline()) f.close() The output is:
Nora This is the commencement line of the file.
In contrast, readlines() returns a list with all the lines of the file equally individual elements (strings). This is the syntax:
For instance:
f = open("data/names.txt") print(f.readlines()) f.shut() The output is:
['Nora\n', 'Gino\n', 'Timmy\due north', 'William'] Notice that in that location is a \northward (newline character) at the end of each string, except the last i.
💡 Tip: Yous can get the same list with listing(f).
You can work with this list in your program past assigning it to a variable or using it in a loop:
f = open("data/names.txt") for line in f.readlines(): # Do something with each line f.close() Nosotros tin can as well iterate over f directly (the file object) in a loop:
f = open("data/names.txt", "r") for line in f: # Practice something with each line f.close() Those are the master methods used to read file objects. At present permit'southward see how you lot can create files.
🔹 How to Create a File
If you need to create a file "dynamically" using Python, you tin do it with the "x" mode.
Permit'southward see how. This is the bones syntax:
Hither's an example. This is my electric current working directory:
If I run this line of code:
f = open("new_file.txt", "x") A new file with that name is created:
With this way, you can create a file and then write to it dynamically using methods that yous will learn in merely a few moments.
💡 Tip: The file will be initially empty until you change it.
A curious thing is that if you lot attempt to run this line again and a file with that name already exists, you will see this error:
Traceback (most recent call last): File "<path>", line 8, in <module> f = open up("new_file.txt", "x") FileExistsError: [Errno 17] File exists: 'new_file.txt' According to the Python Documentation, this exception (runtime mistake) is:
Raised when trying to create a file or directory which already exists.
At present that you know how to create a file, allow'due south run into how yous tin can modify it.
🔸 How to Alter a File
To modify (write to) a file, you lot need to use the write() method. Yous have two ways to do it (append or write) based on the mode that yous choose to open up it with. Allow'southward see them in detail.
Append
"Appending" ways adding something to the finish of some other affair. The "a" mode allows you to open a file to append some content to it.
For instance, if we have this file:
And we want to add a new line to it, we tin can open information technology using the "a" mode (append) and and so, phone call the write() method, passing the content that we want to suspend equally argument.
This is the bones syntax to phone call the write() method:
Here'southward an instance:
f = open("data/names.txt", "a") f.write("\nNew Line") f.close() 💡 Tip: Observe that I'm adding \n before the line to signal that I want the new line to appear as a separate line, not equally a continuation of the existing line.
This is the file now, after running the script:
💡 Tip: The new line might not be displayed in the file until f.close() runs.
Write
Sometimes, y'all may want to delete the content of a file and replace it entirely with new content. You can do this with the write() method if you lot open the file with the "w" way.
Here we have this text file:
If I run this script:
f = open("data/names.txt", "w") f.write("New Content") f.shut() This is the outcome:
As you can see, opening a file with the "w" mode and then writing to it replaces the existing content.
💡 Tip: The write() method returns the number of characters written.
If you want to write several lines at once, you tin can use the writelines() method, which takes a listing of strings. Each string represents a line to be added to the file.
Hither's an example. This is the initial file:
If nosotros run this script:
f = open("data/names.txt", "a") f.writelines(["\nline1", "\nline2", "\nline3"]) f.close() The lines are added to the terminate of the file:
Open File For Multiple Operations
At present you know how to create, read, and write to a file, but what if yous want to practice more one thing in the same program? Permit's see what happens if we try to do this with the modes that you have learned so far:
If you open up a file in "r" mode (read), and then try to write to information technology:
f = open("data/names.txt") f.write("New Content") # Trying to write f.close() You will become this mistake:
Traceback (almost recent call last): File "<path>", line 9, in <module> f.write("New Content") io.UnsupportedOperation: non writable Similarly, if you lot open a file in "due west" style (write), and then try to read information technology:
f = open up("data/names.txt", "w") print(f.readlines()) # Trying to read f.write("New Content") f.close() Y'all will run into this error:
Traceback (most recent call concluding): File "<path>", line fourteen, in <module> print(f.readlines()) io.UnsupportedOperation: not readable The same volition occur with the "a" (suspend) style.
How tin can we solve this? To be able to read a file and perform another operation in the aforementioned program, yous need to add together the "+" symbol to the mode, like this:
f = open("data/names.txt", "westward+") # Read + Write f = open up("data/names.txt", "a+") # Read + Append f = open("information/names.txt", "r+") # Read + Write Very useful, right? This is probably what you will use in your programs, but be sure to include merely the modes that you demand to avert potential bugs.
Sometimes files are no longer needed. Let'southward run into how you tin delete files using Python.
🔹 How to Delete Files
To remove a file using Python, you need to import a module called os which contains functions that interact with your operating system.
💡 Tip: A module is a Python file with related variables, functions, and classes.
Specially, you demand the remove() function. This function takes the path to the file as argument and deletes the file automatically.
Let'due south see an example. We want to remove the file chosen sample_file.txt.
To practice it, we write this code:
import bone os.remove("sample_file.txt") - The start line:
import osis called an "import statement". This statement is written at the top of your file and information technology gives you access to the functions defined in theosmodule. - The second line:
os.remove("sample_file.txt")removes the file specified.
💡 Tip: you lot can employ an absolute or a relative path.
Now that y'all know how to delete files, let'due south encounter an interesting tool... Context Managers!
🔸 Run into Context Managers
Context Managers are Python constructs that will brand your life much easier. Past using them, you don't need to call back to shut a file at the finish of your program and you have access to the file in the particular part of the program that you choose.
Syntax
This is an instance of a context director used to work with files:
💡 Tip: The torso of the context manager has to be indented, merely like we indent loops, functions, and classes. If the code is not indented, it will not exist considered part of the context manager.
When the torso of the context manager has been completed, the file closes automatically.
with open("<path>", "<fashion>") as <var>: # Working with the file... # The file is closed here! Example
Here's an instance:
with open("data/names.txt", "r+") equally f: impress(f.readlines()) This context managing director opens the names.txt file for read/write operations and assigns that file object to the variable f. This variable is used in the body of the context manager to refer to the file object.
Trying to Read it Again
After the body has been completed, the file is automatically closed, so information technology tin't be read without opening it once more. But await! We have a line that tries to read it again, right here below:
with open("data/names.txt", "r+") every bit f: print(f.readlines()) impress(f.readlines()) # Trying to read the file again, outside of the context manager Let's see what happens:
Traceback (most contempo call last): File "<path>", line 21, in <module> print(f.readlines()) ValueError: I/O operation on closed file. This error is thrown because we are trying to read a closed file. Crawly, right? The context director does all the heavy work for us, it is readable, and concise.
🔹 How to Handle Exceptions When Working With Files
When you're working with files, errors can occur. Sometimes you may non have the necessary permissions to modify or access a file, or a file might not fifty-fifty exist.
As a developer, you demand to foresee these circumstances and handle them in your plan to avoid sudden crashes that could definitely affect the user experience.
Let's run into some of the well-nigh mutual exceptions (runtime errors) that you lot might detect when y'all work with files:
FileNotFoundError
According to the Python Documentation, this exception is:
Raised when a file or directory is requested but doesn't be.
For instance, if the file that you're trying to open up doesn't exist in your current working directory:
f = open("names.txt") Yous will see this error:
Traceback (most recent call final): File "<path>", line 8, in <module> f = open("names.txt") FileNotFoundError: [Errno ii] No such file or directory: 'names.txt' Let's interruption this mistake downward this line by line:
-
File "<path>", line eight, in <module>. This line tells you that the error was raised when the code on the file located in<path>was running. Specifically, whenline viiiwas executed in<module>. -
f = open("names.txt"). This is the line that caused the error. -
FileNotFoundError: [Errno 2] No such file or directory: 'names.txt'. This line says that aFileNotFoundErrorexception was raised because the file or directorynames.txtdoesn't exist.
💡 Tip: Python is very descriptive with the error messages, right? This is a huge advantage during the procedure of debugging.
PermissionError
This is another mutual exception when working with files. Co-ordinate to the Python Documentation, this exception is:
Raised when trying to run an functioning without the adequate access rights - for instance filesystem permissions.
This exception is raised when you are trying to read or change a file that don't take permission to access. If you try to do and then, you will meet this error:
Traceback (well-nigh recent call last): File "<path>", line eight, in <module> f = open("<file_path>") PermissionError: [Errno thirteen] Permission denied: 'data' IsADirectoryError
According to the Python Documentation, this exception is:
Raised when a file operation is requested on a directory.
This particular exception is raised when you effort to open or piece of work on a directory instead of a file, so exist really careful with the path that yous pass as argument.
How to Handle Exceptions
To handle these exceptions, you tin use a try/except statement. With this argument, you can "tell" your program what to do in case something unexpected happens.
This is the bones syntax:
endeavor: # Effort to run this code except <type_of_exception>: # If an exception of this type is raised, terminate the process and jump to this block Hither y'all tin can see an example with FileNotFoundError:
effort: f = open("names.txt") except FileNotFoundError: impress("The file doesn't exist") This basically says:
- Try to open the file
names.txt. - If a
FileNotFoundErroris thrown, don't crash! Simply print a descriptive statement for the user.
💡 Tip: Y'all can choose how to handle the situation by writing the appropriate code in the except block. Possibly you could create a new file if it doesn't exist already.
To close the file automatically after the task (regardless of whether an exception was raised or not in the try cake) you tin add the finally block.
endeavour: # Try to run this code except <exception>: # If this exception is raised, stop the process immediately and leap to this block finally: # Do this after running the code, even if an exception was raised This is an example:
endeavour: f = open up("names.txt") except FileNotFoundError: print("The file doesn't exist") finally: f.shut() In that location are many ways to customize the endeavour/except/finally statement and you can even add an else block to run a cake of lawmaking only if no exceptions were raised in the try block.
💡 Tip: To learn more near exception handling in Python, yous may like to read my commodity: "How to Handle Exceptions in Python: A Detailed Visual Introduction".
🔸 In Summary
- You tin can create, read, write, and delete files using Python.
- File objects have their own ready of methods that you tin can use to work with them in your program.
- Context Managers assist yous piece of work with files and manage them past closing them automatically when a task has been completed.
- Exception handling is cardinal in Python. Common exceptions when you are working with files include
FileNotFoundError,PermissionErrorandIsADirectoryError. They can be handled using try/except/else/finally.
I really hope you lot liked my article and found it helpful. Now you tin piece of work with files in your Python projects. Check out my online courses. Follow me on Twitter. ⭐️
Learn to code for free. freeCodeCamp's open source curriculum has helped more than twoscore,000 people get jobs as developers. Get started
Source: https://www.freecodecamp.org/news/python-write-to-file-open-read-append-and-other-file-handling-functions-explained/
0 Response to "Read Files From Data Lake Using Python"
Post a Comment