This tutorial explains how to create a project in Scrapy.
Starting a Project in Scrapy can be a little confusing for beginners. Hence we’ll be clearing away any doubts in this detailed tutorial on Project Creation in Scrapy.
Setting up a Scrapy Project
We’ll be using the PyCharm IDE to create our Scrapy project. It’s a pretty good and popular IDE and makes managing Scrapy Projects quite easy. You can still follow this tutorial though and replicate the same steps on other IDE’s like IntelliJ.
Open up the IDE of your choice and then create a new project. You can find the “New Project” button in the file menu usually. The name of our Scrapy project is
Next run the following line of code in the command terminal of the IDE to begin a scrapy project. The command terminal is typically located near the bottom left.
Once this command has been run, a folder with the same name as the project (tutorial in this case) will be created. It will contain the same files as shown in the image below. You can choose whatever name you want., just keep it meaningful.
The Spiders folders is where we’ll be creating all our “Spiders”. Our main code will be written in this folder. Right click on the spiders folder and create a new file as shown in the image below.
Once we’ve reached this point, we can finally begin writing scrapy code. Remember to select the
example.py file and open it up in the main window. We’ll be writing our main code for our Python Scrapy tutorial here.
You can create more than just one Spider in the spiders folder. Just create another file using the new file option, but with a different name.
This marks the end of the “Create a Project in Scrapy” Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the article content can be asked in the comments section below.