import requests from bs4 import BeautifulSoup as bs import re url = "https://matlabhelpers.com/" page = requests.get(url) soup = bs(page.content , 'html.parser') head = soup.find('head') # re.findall(r'[^<]+', str(head)) #SSL secure ssl = soup.find(attrs={"rel": re.compile(r'canonical', re.I)}) # ssl['href'] if re.search('(https:)', ssl['href']): print('SSL Secure!!!') else: print('Bad news your website is not SSL secure')
Tag: web scraping
.strip() , .replace() and findAll()
Today's topic is .replace(), .strip() and findAll() *****Learn everyday to grow everyday***** This three are very useful function for beginners and for web scrapers while they are scraping a webpage. .strip() .strip() is used to remove white space from the start and end of the string. ## .rstrip() and .lstrip() .rstrip() is used … Continue reading .strip() , .replace() and findAll()
WEB SCRAPING(scrap h1 tag from list of urls)
For detailed explanation and code click here - https://github.com/theone9807/Webscraping-scrap-h1-tag-from-list-of-urls- #made a web scrapper which scrapped H1 from a page . #And it is able to scrap from a list of urls and save it in a data frame. In [1]: import pandas as pd df = pd.read_excel('test.xlsx', sheetname=0) # can also index sheet by name … Continue reading WEB SCRAPING(scrap h1 tag from list of urls)
thefuture 