File Handling is tricky business. Once you begin using File Handling on a larger scale in bigger applications you’ll come across many obstacles and complexities. This is why we created the Python Advanced File Handling Guide.
This article will walk you through most of these problems, and teach you how to use File handling for more advanced purposes. Like saving and loading game files, creating log files, saving and loading 2-Dimensional objects to a file and more.
You may want to check out our prequel, the Python File Handling guide before continuing further!
Reading from a File
Reading from Files was already covered in our Basic File Handling guide. This time however, we will be addressing a common problem that occurs while reading data from files. In this section, we’ll discuss the new line character, “\n“.
The newline character is used to mark a newline in a file. We can’t see it, but it’s always there. This is the computer’s way of understanding that there must be a line break between two lines.
Case 1: Using the new line character
Running the following code, results in all three pieces of text written in the same line without any spacing.
file = open("File3.txt", 'w')
file.write("Object1")
file.write("Object2")
file.write("Object3")
file.close()
Output: (As shown in our File3.txt)
Object1Object2Object3Adding a new line character to each string before writing it helps us avoid this problem.
file = open("File3.txt", 'w')
file.write("Object1" + "\n")
file.write("Object2" + "\n")
file.write("Object3" + "\n")
file.close()
New Output: (As shown in File3.txt)
Object1
Object2
Object3You can also write it like this: file.write("Object1\n")
Case 2: Avoiding the new line character
Continuing from the previous example, what would happen if, after writing the output to the file, we tried to read it again? Depending on how you saved the data, there is a chance of the new line character making an appearance when we read the data.
The following code will read the contents of File3.txt and display them.
file = open("File3.txt")
for x in file:
print(x)
file.close()
Output:
Object1
Object2
Object3
As you can see, there is an extra new line between each string. This becomes a problem when reading data from files and storing in lists and arrays. There is an extra newline character stored along with the values.
Why does this happen? Remember that “\n” character we wrote to the file? Basically, that’s what causing the problem. There’s an extra new line character present now. Luckily, there are ways to combat this issue. The goal is to read the data from the file, and be able to store it properly in a list.
The splitlines() function splits the string based on the lines. It breaks the string where it finds a line break (newline character) and returns a list of splitted strings.
file = open("File3.txt")
for x in file:
print(x.splitlines())
file.close()
Output:
['Object1']
['Object2']
['Object3']As you can see, there are no longer any gaps in between them. We’re free to use these values further on in our program now. Not to say that this is the only way of getting past this issue. For one, you can try the split() function, or you can write code that skips past all blank lines. Let us know what worked best for you.
Creating a log file
To demonstrate why and how we use log files, we’re using the code for a sorting algorithm called bubble sort. Don’t worry if you don’t understand the bubble sort code. For this section, all you need to know is that the bubble sort algorithm is used to sort out a list of items from smallest to largest (Ascending order).
We will be using the log file to record the state of the “mylist” list, after each iteration. This allows us to record each individual time it was changed.
def bubblesort(list):
index = len(mylist)
n = index - 1
for i in range(index - 1):
for j in range(n):
if mylist[j] > mylist[j+1]:
mylist[j], mylist[j+1] = mylist[j+1], mylist[j]
File.write(str(mylist) + "\n")
n = n - 1
File = open("log.txt", 'w')
mylist = [3,1,9,5,3,7,2]
bubblesort(mylist)
File.close()
Output: (Taken from the log.txt file)
[1, 3, 9, 5, 3, 7, 2]
[1, 3, 5, 9, 3, 7, 2]
[1, 3, 5, 3, 9, 7, 2]
[1, 3, 5, 3, 7, 9, 2]
[1, 3, 5, 3, 7, 2, 9]
[1, 3, 3, 5, 7, 2, 9]
[1, 3, 3, 5, 2, 7, 9]
[1, 3, 3, 2, 5, 7, 9]
[1, 3, 2, 3, 5, 7, 9]
[1, 2, 3, 3, 5, 7, 9]This was just a example demonstration. It is in similar conditions that log files are used in the real world. Log files record and track the states of variables and objects, noting down each individual change in them and “when” the change occurred.
All this is to make the Debugging and testing process easier. If you have a log file, you can go through it examining the whole process and eventually narrow down the problem.
File handling in 2 Dimensions
We’ll be simultaneously dealing with two sections here. First, the loading and saving of game files, and using 2-D objects like Arrays and Lists in File handling. We’ll be using an RPG game setting to demonstrate. This is the most difficult part of our Python Advanced File handling guide, accumulating everything we have taught about File handling here.
To start off, this is our game file (shown below). It “Tracks” the various entities and their status. The format is as follows. First Name of entity, followed by HP, Item equipped, Attack and Defense respectively. We’ll be working with this file for the remainder of this section.
Tracker = [["Player", "Sword", "20","15"],
["Enemy1", "Knife", "15", "8"],
["Enemy2", "Branch", "5", "7"],
["Enemy3", "Spear", "12", "10"]]Saving the File
Saving the File is by far the easier part. We were able to find the number of rows of finding the length of Tracker. The length of a 2-D object is always the number of objects in it. In this case, there are 4 Lists in Tracker. To find the number of columns, we simply find the length of one of the 4 lists within Tracker. The length of one List happens to be 4 as well. Hence, that’s our number of columns.
file = open("Save.txt", 'w')
rows = len(Tracker)
cols = len(Tracker[0])
for i in range(rows):
for j in range(cols):
file.write(Tracker[i][j] + "\n")
file.close()
Loading the File
This is the most complex part, involving a number of calculations before we actually get the main part.
- Step one, counting the total number of lines in the File. It’s a simple loop that runs through each line, incrementing a variable each time by 1. The loops ends when the last line is read.
- Step two, finding the number of rows. While the numbers of rows may vary, the number of columns will not. Hence, using simple mathematics we can find the number of rows. An Alternate way of doing this is including the number of rows and columns when saving the file. Then when loading the file you can read the lines containing the rows and columns and adjust accordingly.
- The lists must first be initialized.
- The variable
xstores an individual element from a list, like “Spear” or “15”. We usesplitlines()to get rid of the newline character. This also convertsxinto a list object. From here simply concatenate every 4xvalues, creating one list which we append toTracker. The value oftemplistis reset after the completion of one list, so that it’s ready for the next.
#Counts the number of lines in the File
file = open("Save.txt", 'r')
count = 0
for x in file:
count = count + 1
file.close()
#Calculating number of rows (Total lines / columns)
rows = int(count / 4)
#Initializing Lists
Tracker = []
templist = []
#The actual code
file = open("Save.txt",'r')
for i in range(rows):
for j in range(4):
x = file.readline()
templist = templist + x.splitlines()
print(templist)
Tracker.append(templist)
templist = []
print(Tracker)
file.close()
Output:
['Player', 'Sword', '20', '15']
['Enemy1', 'Knife', '15', '8']
['Enemy2', 'Branch', '5', '7']
['Enemy3', 'Spear', '12', '10']
[['Player', 'Sword', '20', '15'], ['Enemy1', 'Knife', '15', '8'], ['Enemy2', 'Branch', '5', '7'], ['Enemy3', 'Spear', '12', '10']]As you can see, it’s a perfect recreation. From the save file we were able to perfectly retrieve each individual entity and recreate the Tracker 2-D list.
Note: This would actually be simpler using 2-D arrays. However, I decided to go with lists instead as Arrays require the Python Library Numpy, which less people would be familiar with. The concept remains roughly the same however.
This marks the end of our Python Advanced File Handling Guide. Any suggestions or contributions for our site Coderslegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.