Enable X/Twitter accounts to start scraping

This guide will help you configure your Masa Node as a X/Twitter scraper using user/pass authentication.

An X/Twitter Pro Account is required

A paid X/Twitter Pro Account is absolutely necessary for scraping X/Twitter data. Ensure you have obtained a paid Twitter Pro Account before proceeding with the configuration.

1

Step 1

Set environment variable to enable X/Twitter scraping

Add the following lines to your .env file to enable X/Twitter scraping:

TWITTER_SCRAPER=true
2

Step 2

Enter Your X/Twitter Account login credentials in the .env file

Add the following line to your .env file, replacing username1:password1,username2:password2 with your actual X/Twitter account credentials:

TWITTER_ACCOUNTS=username1:password1,username2:password2

It is highly recommended to use multiple accounts to avoid being rate limited or banned by X/Twitter. We recommend a minimum of 3. Some top miners have up to 16 or more.
Note: Use multiple accounts to avoid rate limiting or bans. Rate limiting starts at 450 tweets. Separate each account with a comma.

Do not use Twitter accounts that you care about, since there is a small risk of them being suspended.

3

Step 3

Configure multiple user agents

Add the following line to your .env file to specify multiple user agents. This helps the scraper mimic different browsers and increase login success rates.

USER_AGENTS="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36,Mozilla/5.0 (Macintosh; Intel Mac OS X 14.7; rv:131.0) Gecko/20100101 Firefox/131.0"
4

Step 4

Restart your miner node

Stop the Masa node by pressing Ctrl + C in the terminal where it is running, and then restart the miner node to apply the changes. Run the miner node

To start the miner node, follow these steps:

  1. Open your terminal.
  2. Navigate to the directory where your miner node is located.
  3. Run the following command to start the miner node:
make run

The miner node will start and begin processing. You should see logs indicating that the node is running and processing data.

5

Step 5

Verify that the X/Twitter scraper is enabled

Check the logs after running the miner node to verify that the X/Twitter scraper is enabled.

#######################################
#     __  __    _    ____    _        #
#    |  \/  |  / \  / ___|  / \       #
#    | |\/| | / _ \ \___ \ / _ \      #
#    | |  | |/ ___ \ ___) / ___ \     #
#    |_|  |_/_/   \_\____/_/   \_\    #
#                                     #
#######################################

Is TwitterScraper:   True
Is DiscordScraper:   false
Is TelegramScraper:  false