scrape_urls

Functions

find_html_class(soup, class_name)

Finds all elements with a given class name.

find_id_in_html(soup, id)

Finds an element with a given id in a BeautifulSoup object.

find_in_html(soup, element)

Finds an element in a BeautifulSoup object.

get_table(soup[, table_num, row_start, row_end])

Pulls out a table from a beautifulsoup html.

multi_thread_func(func, values[, threads])

This function takes a function and a list of values.

proxy_generator()

This function scrapes a list of a free proxies from:

scrape_page(url[, spoof])

This function tries to get page information by spoofing the header and trying a random proxy.