Bash Script Installation and Setup Guide

Prerequisites for Installation and Execution

To run the provided Bash script on Linux or Termux, you need to fulfill the following prerequisites:

For Linux

Operating System: Ensure you are using a Linux distribution (e.g., Ubuntu, Debian, CentOS).

Required Packages: You need to install the following packages:

sudo apt-get update

sudo apt-get install curl

sudo apt-get install bash

sudo apt-get install grep sed awk

Script Permissions: Set executable permissions for the script file:
```
chmod +x script.sh
```

For Termux (Android)

Installation: First, install Termux from the Google Play Store or F-Droid.
Update and Upgrade: Update the package list and upgrade installed packages:
```
pkg update && pkg upgrade
```

Required Packages: Install necessary packages:

pkg install curl

pkg install bash

pkg install grep sed gawk

Script Permissions: Grant execution permissions to the script:
```
chmod +x script.sh
```

Script Code


#!/bin/bash

# Script to perform automated Google searches, extract links, and fetch their content.
# Includes simulated QoS, ToS, and CoS headers for testing purposes with a realistic referer.

# Define the list of search keywords
keywords=(
    "IT" "ICT" "OT" "IIOT" "IOT" "network" "cybersecurity" "AI" "machine+learning" "data+science"
    "cloud+computing" "blockchain" "automation" "digital+transformation" "IoT" "big+data" 
    "analytics" "software+development" "IT+consulting" "networking" "virtualization" 
    "system+integration" "tech+trends" "IT+strategy" "smart+devices" "enterprise+IT" 
    "cyber+defense" "data+protection" "IT+infrastructure" "technology+solutions" 
    "security+services" "cloud+storage" "IT+support" "tech+innovation" "software+engineering" 
    "information+security" "IT+management" "digital+marketing" "IT+services" "enterprise+solutions" 
    "IT+architecture" "IT+operations" "mobile+computing" "IT+project+management" "IT+training" 
    "tech+consulting" "network+security" "IT+systems" "data+analytics" "IT+compliance" 
    "IT+governance" "IT+trends" "IT+support+services" "IT+outsourcing" "technology+consulting"
)

# Define a list of realistic referers based on common sources
referers=(
    "https://www.google.com/search?q="
    "https://www.bing.com/search?q="
    "https://www.yahoo.com/search?p="
    "https://www.duckduckgo.com/?q="
    "https://www.ask.com/web?q="
)

# Function to generate a random IPv4 address
generate_random_ipv4() {
    echo "$((RANDOM % 256)).$((RANDOM % 256)).$((RANDOM % 256)).$((RANDOM % 256))"
}

# Function to generate a random IPv6 address
generate_random_ipv6() {
    echo "$(printf '%x:%x:%x:%x:%x:%x:%x:%x' $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)) $(($RANDOM % 0xffff)))"
}

# Function to generate a random User-Agent string
generate_random_user_agent() {
    local user_agents=(
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/91.0.864.48 Safari/537.36"
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Firefox/89.0"
        "Mozilla/5.0 (Android 11; Mobile; rv:90.0) Gecko/90.0 Firefox/90.0"
        "Mozilla/5.0 (Android 10; Mobile; rv:89.0) Gecko/89.0 Firefox/89.0"
    )
    echo "${user_agents[RANDOM % ${#user_agents[@]}]}"
}

# Function to generate a realistic referer URL
generate_realistic_referer() {
    local keyword="$1"
    echo "${referers[RANDOM % ${#referers[@]}]}${keyword}"
}

# Function to generate random QoS, ToS, and CoS values
generate_random_qos_tos_cos() {
    local qos=$((RANDOM % 8))        # QoS: 0-7
    local tos=$((RANDOM % 256))      # ToS: 0-255
    local cos=$((RANDOM % 8))        # CoS: 0-7
    echo "$qos,$tos,$cos"
}

# Function to perform a search and process links
perform_search_and_process_links() {
    local keyword="$1"
    local search_url="https://www.google.com/search?q=${keyword}+site:miralishahidi.ir"
    
    # Randomly select HTTP version (1.1 or 2)
    local http_version=$(( RANDOM % 2 + 1 ))
    local http_version_flag="--http1.1"
    if [ "$http_version" -eq 2 ]; then
        http_version_flag="--http2"
    fi

    # Randomly generate User-Agent, Referer, and IP
    local user_agent=$(generate_random_user_agent)
    local referer=$(generate_realistic_referer "$keyword")
    local ip_version=$(( RANDOM % 2 ))
    local forwarded_ip
    if [ "$ip_version" -eq 0 ]; then
        forwarded_ip=$(generate_random_ipv4)
    else
        forwarded_ip=$(generate_random_ipv6)
    fi
    
    # Generate random QoS, ToS, and CoS values
    local qos_tos_cos=$(generate_random_qos_tos_cos)
    IFS=',' read -r qos tos cos <<< "$qos_tos_cos"

    echo "Fetching page content from: $search_url with HTTP version: $http_version_flag"
    echo "Using X-Forwarded-For IP: $forwarded_ip"
    echo "QoS: $qos, ToS: $tos, CoS: $cos"
    echo "Referer: $referer"
    
    # Fetch the page content with random headers
    page_content=$(curl -s -A "$user_agent" -H "X-Forwarded-For: $forwarded_ip" -H "Referer: $referer" -H "QoS: $qos" -H "ToS: $tos" -H "CoS: $cos" $http_version_flag "$search_url")
    
    # Check if the page content was fetched successfully
    if [ -z "$page_content" ]; then
        echo "Failed to fetch page content. Skipping..."
        return
    fi
    
    # Extract and clean up links
    echo "Extracting and cleaning up links..."
    echo "$page_content" | grep -oP '(?<=href="/url\?q=)[^"]*' | \
        sed 's/%3A/:/g; s/%2F/\//g; s/%3F/?/g; s/%26/&/g; s/%2C/,/g; s/%2B/+/g; s/%20/ /g' | \
        awk -F'&' '{print $1}' | \
        sed 's/%20/ /g; s/%21/!/g; s/%2A/*/g; s/%28/(/g; s/%29/)/g; s/%7E/~/g; s/%2D/-/g' | \
        grep '\.html$' | while IFS= read -r link; do
            echo "Extracted link: $link"
            
            # Fetch the link content and measure time taken
            echo "Fetching content from: $link with HTTP version: $http_version_flag"
            
            response=$(curl -s -A "$user_agent" -H "X-Forwarded-For: $forwarded_ip" -H "Referer: $referer" -H "QoS: $qos" -H "ToS: $tos" -H "CoS: $cos" $http_version_flag -w "%{http_code}" -o /tmp/link_content.html "$link")
            
            http_status=$(echo "$response" | tail -n 1)
            if [ "$http_status" -ne 200 ]; then
                echo "Failed to fetch link content. HTTP status code: $http_status"
                continue
            fi

            echo "HTTP Status Code: $http_status"
            
            echo "Link content preview:"
            head -n 10 /tmp/link_content.html
            
            echo "----------------------------------------"
        done
    
    echo "Link processing complete for keyword: $keyword"
}

# Function to get a random keyword from the list
get_random_keyword() {
    echo "${keywords[RANDOM % ${#keywords[@]}]}"
}

# Main script execution

# Perform searches and process links
echo "Starting Google searches and link processing..."
while true; do
    keyword=$(get_random_keyword)
    perform_search_and_process_links "$keyword"
    sleep 10 # Sleep for 10 seconds to prevent overloading the server
done

echo "All tasks completed."

Detailed Execution Instructions

Step-by-Step Execution

Download the Script: Ensure the script is saved with an appropriate name, e.g., script.sh.
Set Executable Permissions: Make the script executable by running:
```
chmod +x script.sh
```
Run the Script: Execute the script with:
```
./script.sh
```
Continuous Execution: The script runs indefinitely. To stop it, use Ctrl+C to interrupt the process.
Monitor Output: Observe the console output for status messages, extracted links, and content previews.

Package and Library Requirements

Before running the script, ensure that you have the following packages installed:

For Linux

curl: Required for fetching web pages.
grep: Used for pattern searching in text.
sed: Stream editor for filtering and transforming text.
awk: Text processing tool.

For Termux

curl: Install with pkg install curl
grep: Install with pkg install grep
sed: Install with pkg install sed
awk: Install with pkg install gawk

Execution Details

How the Script Works

Initialization: The script initializes with a set of predefined keywords and user agents.
Generating Random Values: It generates random User-Agent strings, IP addresses, and QoS, ToS, CoS values to simulate real-world browsing.
Performing Searches: The script performs searches on Google for each keyword.
Fetching and Processing Content: It fetches page content, extracts and cleans up links, and processes each link to fetch its content.
Handling Errors: If fetching fails, the script logs the error and continues with the next link.