Try UiPath Free

5 October 2020

Contact your Local Representatives by Building a Data-Scraping Robot in UiPath Studio

5 October 2020

Contact your Local Representatives by Building a Data-Scraping Robot in UiPath Studio

⬅  SEE ALL THE ARTICLES

 

Introduction

 

With everything that has been going on in the United States (U.S.), now (more than ever) people are looking for ways to contact their government representatives. While emails and phone calls are popular, more and more people are turning to mailing in letters to their representatives. However, one of the most time-consuming parts of sending these letters is gathering the mailing addresses for all the different parties.

 

Here, a UiPath robot:

1. Navigates to a website

2. Scrapes all of the website data related to a specific zip code

3. Sends an email with the addresses listed for most city, county, and state officials

 

By doing the leg work of pulling this information, the robot makes contacting your U.S. politicians even easier!

 

In this tutorial, you’ll learn how to build this data-scraping robot. And hopefully when you’re done you can use this robot to help your friends and family get their representatives’ mailing addresses too! The robot can’t write the letters for you—that’s for you to decide what you want to see improved and changed in the world!

 

Requirements

 

Before getting started, you’ll need to do the following:

- Have internet access

- Download and install UiPath Studio

- Download and log into Microsoft Office

 

Step 1: Create a new project in UiPath Studio

When you open UiPath Studio, create a new Process.

 

Picture1-Sep-22-2020-04-53-21-84-PM

 

Click on Process and let’s name it Politician_Address_Bot

 

Picture2-2

 

Then click the ‘Create’ button.

 

Step 2: Click on ‘Open Main Workflow’

 

Picture3-1

 

Step 3: Drag in our first activity ‘Input Dialog’ to collect some data

On the left-hand side, find your Activity pane/window and search for ‘Input Dialog’

 

Picture4

 

Then, click and drag the ‘Input Dialog’ activity from the activity pane into the middle design window.

 

Picture5

 

**Note** We are using input dialogues to collect the two required fields for this process - a zip code and an email address. If you feel comfortable, you can collect these two fields in a manner of different ways, but to keep it simple for the tutorial, we are using ‘Input Dialog’ activities.

 

Step 4: Let’s update the ‘Input Dialog’ box we dragged in and ask our users for input

Your design window should now look like this:

 

Picture6

 

We are going to provide values for both the Title and Label fields. You can type whatever you would like into each of these fields, but make sure the text you type goes between double quotes.

 

The title is the title of the window that will pop up to the user and the label is text that will appear on the box. For example, here is what I recommend typing:

 

Picture7-1

 

At this point, you should be able to hit the big Play button in the top left corner and you’ll see this pop up on your screen!

 

Play button:

 

Picture8

 

What you should see when you hit the play button:

 

Picture9

 

If you see that, great, your robot is running! For now, let’s go ahead and click ‘Ok’ to close the box. When you hit ‘Ok,’ you will close the box, end the process, and your robot will stop running.

 

Step 5: Let’s collect input from our ‘Input Dialog’

Now, making sure that you have clicked on the ‘Input Dialog’ activity you dragged in (you should see the activity/box highlighted in blue), on the right-hand side of your screen click on the ‘Properties’ pane.

 

Picture10

 

Once open, it should look like this:

 

Picture11-1

 

Our Properties pane allows us to customize each activity further! If you don’t see this window, please make sure that you have selected the ‘Input Dialog’ box or you will not see the Properties pane for that activity.

 

Take note of the ‘Output’ section at the bottom of your properties pane. Click into the box adjacent to ‘Result:’

 

Picture12-1

 

Once you have clicked in the box, type ‘Ctrl + k’ into the box in order to create a new variable. We use variables to store data. In this case, we want to store the zip code the user enters into a variable. You should see this:

 

Picture13

 

The ‘Set Var’ that shows up allows us to create a variable in the box. Once you see ‘Set Var,’ type in ‘zipCode’ and then hit the Enter key.

 

Picture14

 

Congrats! You have made your first variable!

 

Step 6: Let’s repeat this process of creating the ‘Input Dialog box, but this time for an email address!

To repeat this process, refer to the prior steps. When you drag in the next ‘Input Dialog’ activity, you need to drag it below the ‘Input Dialog’ box you already dragged in. It should look like this:

 

Picture15

 

Repeat the whole process but change your input box text to be relevant to an email address. And when you set this Output variable, make sure you set it to the name ‘emailAddress’.

 

Like so:

Picture16

 

Step 7: Drag in our next activity - ‘Open Browser’

On the left-hand side, find your Activity pane/window and search for ‘Open Browser.’

 

Picture17

 

Then, click and drag ‘Open Browser’ from the activity pane into the middle design window BELOW the last ‘Input Dialog’ activity.

 

Picture18

 

Step 8: In the ‘Open Browser’ activity, enter in this URL: "https://myreps.datamade.us/"

Make sure the URL is in quotes:

 

Picture19

 

If you want to test, at this point you can run your program by clicking the Play button in the top left corner of UiPath Studio. You should be brought to this page (below) after the two input boxes have shown up:

 

Picture20

 

Step 9: Drag in a ‘Type Into’ activity to the main window in the “Do” box.

Search for ‘Type Into’

 

Picture21

 

Drag into the ‘Do’ box.

 

Picture22

 

Step 10: Click ‘Indicate element inside browser’ and indicate the box to type into on the screen

After you click on ‘Indicate element inside browser,’ you’ll be prompted to view whatever window is open behind UiPath Studio. Ensure that you have that webpage open and it’s the window behind UiPath Studio.

 

You’ll then see wherever you hover your mouse will show a blue box being highlighted. The boxes are all the things you can interact with on the page. Hover over the input box on the screen like so:

 

Picture23 (2)

 

Step 11: In the ‘Type Into’ activity, enter in (no quotes) ‘zipCode’

We are leveraging the zipCode variable we created earlier in order to use the entered value in our automation.

 

Picture24

 

Step 12: In this ‘Do’ box, drag in a ‘Click’ activity after the ‘Type Into’

Make sure the boxes are ordered as shown here:

 

Picture26

 

Step 13: Just like we indicated the textbox on our screen, we are going to hover over the ‘Search’ button and click to indicate like we did with ‘Type Into’

Make sure the boxes are ordered as shown here:

 

Picture27

 

Great! You’ve just clicked on a button using UiPath! If you hit the ‘Run’ button from here, and provide input values for your dialogue boxes, you should see a zip code type into the box, and then the search button clicked.

 

Make sure you’re keeping track of your open Windows and closing them after testing. We’ll have the robot close them for us later.

 

Step 14: Scrape the city data

Make sure the boxes are ordered as shown. Using ‘77008’ as our testing zip code and ‘test@uipath.com’ as our testing email address you should see this screen on the website:

 

Picture28

 

Scrolling down, you will see three tables, one for ‘Houston’ city, one for ‘Harris County,’ and one for ‘Texas.’ This website presents those three levels of government to the user. In some cases, only county and state information are presented, but we’ll handle that later.

 

We will be scraping data three different times. One for each table. For this step, it is crucial we pay attention to where we are adding in the next activities. By now, you may have noticed we build through a series of nested boxes and this ordering becomes very important down the road.

 

Before you complete the next step, make sure you have clicked in the area below your ‘Click’ activity. That will ensure that the next step gets added in the correct place.

 

To start, click the ‘Data Scrape’ button in the top left corner of Studio.

 

Picture29

 

You will see this wizard pop up:

 

Picture30

 

We will be interacting with this screen and the ‘myreps’ website. Make sure the website is pulled up immediately behind this window. Click the ‘Next’ button.

 

You will then be prompted with the blue boxes again on hover. Hover over the first cell in the ‘City’ table and click.

 

Picture31

 

After you click, you’ll now see this window (below) asking if you want to grab all of the table data. Click ‘Yes.’

 

Picture32

 

You should then see this with all the data displayed from the ‘City’ table. Click ‘Finish,’ and it will store the data.

 

Picture33

 

You will see this window:

 

Picture34

 

For now, click ‘No’ and we will be done with the data scraping wizard. Studio should look like this now in the design pane:

 

Picture35

 

Step 15: Write data to a Microsoft Excel sheet

Search for a ‘Write Range’ in the Activities pane. Be careful, here you will have two options. We are going to use the option listed under ‘Workbook:’

 

Picture36

 

Write your ‘Write Range’ activity so that it looks like this:

 

Picture37

 

Let’s break this down. Above, in the first input box, we are providing a name for our Microsoft Excel sheet. We used the data stored in the variable ‘zipCode’ and the written-out text ‘CityData.xlsx’. By using a combination of text in double quotes and a variable, when we create the Excel sheet it will be named:

“77008CityData.xlsx”

 

The second input box ‘Sheet1’, tells us which sheet to write the data to, or which sheet to create.

 

The third input box ‘A1’ tells us which cell to start writing the data into. Think of this as the upper left most cell of where your data will be within your spreadsheet.

 

The last input box ‘ExtractDataTable’ is our variable that holds the data we extracted. The robot automatically generated that variable as a result of the data scraping.

 

Lastly, open the Properties pane on the right side of Studio while having the ‘Write Range’ activity selected.

 

Picture38

 

We are going to make sure to check the “AddHeaders” option is checked. And that’s it! It only takes one activity to write data to an Excel sheet.

 

Step 16: Scrape the County data

IMPORTANT: Where this next step gets nested is very important. So, before you click on the ‘Data Scrape’ button in Studio, make sure you have clicked and indicated below the grouping of activities in the prior ‘Data Scrape’ (but still within the prior window). See the image below to see where to click and which box should be highlighted.

 

Picture39

 

By clicking before hitting ‘Data Scrape,’ you will force the ‘Data Scrape’ activities to appear in a specific nested order.

 

For this next ‘Data Scrape,’ we are going to select the first cell in Harris County and repeat the same process.

 

Picture40

 

Once you have completed that, your design pane in Studio should look like this:

 

Picture41

 

Your indication that you did it correctly should be that the two Data Scraping activities show up at the same level within its parent block.

 

Repeat the same steps as above to rename and rescope the result variable, but this time name it CountyDataTable. It should look like this: 

 

Picture42

 

Step 17: Write County data to an Excel sheet

We will repeat step 15 here. Instead of writing the City Data though, we will be writing the County Data to its own Excel sheet.

 

Again, making sure you drag the ‘Write Range’ activity to the right place is crucial! You will place it right below the ‘Data Scrape’ in the ‘Do’ box:

 

Picture43

 

Once you have dragged in your activity, we will populate the fields similar to Step 15. Your completed box should look like this:

 

Picture44

 

It’s very similar to what we did in step 15. Lucky for us, the variable Extract Data Table is created uniquely for each Data Scrape, so even though we are using the same variable, every time we perform a Data Scrape the value gets updated to the current data extracted.

 

Because of this, all we have to do is change the name of the file from ‘City’ to ‘County.’

 

Reminder: Go into the properties pane like you did in step 15 and check the ‘AddHeaders’ box.

 

Step 18: Scrape the State data

We will repeat steps 14 and 15 but this time you will be collecting the State official data. Again, start position is very important. We will again click at the same level as City and County. You should click at the position indicated below before clicking ‘Data Scrape.’

 

Picture45

 

Once clicking there, start the ‘Data Scrape’ process as normal, but for the State data.

 

Picture46

 

Once the data scrape is complete, we again move to update the extracted data variable, update the scope, and rename it to ‘StateDataTable’.

 

Step 19: Write State data to an Excel sheet

We will repeat steps 15 and 17 here. Instead of writing the City or County Data though, we will be writing the State Data to its own Excel sheet.

 

Again, making sure you drag the ‘Write Range’ activity to the right place is crucial! You will place it right below the ‘Data Scrape’ in the ‘Do’ box:

 

Picture47

 

Once you have dragged in your activity, we will populate the fields similar to steps 15 and 17. Your completed box should look like this:

 

Picture48

 

Again, all we have to do is change the name of the file to reflect it’s the State data. Reminder: Go into the properties pane like you did in step 15 and check the ‘AddHeaders’ box.

 

Step 20: Close the website window!!

Now that we have extracted all of our data, it’s good practice to go ahead and exit out of our browser window.

 

After the ‘Extract Data’ table activity in our most recent State ‘Data Scrape’ block, we are going to add a ‘Close Tab’ activity. This will close out of Internet Explorer (IE) for us. It should be placed here:

 

Picture49

 

Step 21: Format email greeting

Now that we have extracted our data it’s time to create a greeting for our email!

 

This part will get tedious. At the end of the day, we are just manipulating some strings. You can improvise and get creative with how you choose to craft your message. This tutorial just shows one possible example. We’ll start by dragging in an Assign activity one level up from the last ‘Data Scrape’ activity:

 

Picture50

 

In the ‘To’ section, type ‘Ctrl+k’ and we’ll create a variable like we did earlier. Let’s name this variable ‘emailBody’.

 

For most of the next steps, instead of editing things in the main design pane, we will be creating our email body by expanding the input box through the properties pane. Clicking the ‘...’ in the ‘Value’ property will open up a window:

 

Picture51

 

You should see the below window when you click ‘...’:

 

Picture52

 

Let’s start our email body off with a greeting. Type what you see in the screenshot below so we have matching boxes:

 

Picture53

 

Let’s break this down. Whenever you type text directly into the box, it needs to be surrounded by quotes.

  1. • ‘Environment.NewLine’ tells the text to line break.
  2. • One ‘Environment.NewLine’ makes a break and moves the text to the next line.
  3. • Using two ‘Environment.NewLine’ provides for a full blank line before your next text is typed.

So, it ends up looking like this:

 

Hello!

Attached is the data for your zip code: 77008

‘+ zipCode’ allows us to use the value stored in zipCode and print the text to the screen. Since zipCode is a variable, it does not need to be typed with quotes around it.

 

Step 22: Create our email activity

The last step in preparing our email is attaching the Excel files we created to the email! Drag in a ‘Send Outlook Mail Message’ below the ‘Assign’ activity:

 

Picture54

 

Now, let’s populate that activity like so:

Picture55

 

We entered in emailAddress without quotes because it represents the variable we created to store the email address the user entered. For the email subject, we manually typed in a subject, so we used double quotes. And for the body of the email, we provide the variable we just defined in the previous ‘Assign’ with the email body.

 

Click the ‘Attach Files’ button in the ‘Send Outlook Mail Message’ activity:

 

Picture56

 

You should then see this window:

 

Picture57

 

Click on ‘Create Argument.’

 

Picture58

 

You will see this screen. Click where indicated on the text ‘Enter a VB expression’ and type in the names of your Excel file! Repeat this two more times to include all of the Excel sheets we’ve made. It should look like this:

 

Picture59

 

What we’re doing here is providing the same names that we used to create the Excel sheets so that we can tell the robot to attach these Excel sheets to the email. Once you have completed this step, you are done!!! You now have a robot that gathers information from a website and then emails the information to users so that they can make the process of letter writing and contacting their politicians as easy as possible!

 

 

Conclusion

 

Now that you have your politician’s addresses, the robot’s work is done. The rest is up to you.

 

Now go out there and advocate for good!

 

And congratulations! You’ve learned how to create variables, scrape data, write data to Microsoft Excel, and send emails using Microsoft Outlook!

 

Christine Moore is a Pre-Sales Engineer at UiPath.


by Christine Moore

TOPICS: Learning RPA, UiPath Studio, RPA Tutorial

Show sidebar