python - How to scrape Instagram with BeautifulSoup -


i want scrape pictures public instagram account. i'm pretty familiar bs4 started that. using element inspector on chrome, noted pictures in unordered list , li has class 'photo', figure, hell -- can't hard scrape findall, right?

wrong: doesn't return (code below) , notice the code shown in element inspector , code drew requests not same aka no unordered list in code pulled requests.

any idea how can code shows in element inspector?

just record, code start, didn't work because unordered list not there:

from bs4 import beautifulsoup import requests import re  r = requests.get('http://instagram.com/umnpics/') soup = beautifulsoup(r.text) x in soup.findall('li', {'class':'photo'}):     print x

thank help.

if @ source code page, you'll see javascript generates webpage. see in element browser webpage after script has been run, , beautifulsoup gets html file. in order parse rendered webpage you'll need use selenium render webpage you.

so, example, how selenium:

from bs4 import beautifulsoup import selenium.webdriver webdriver  url = 'http://instagram.com/umnpics/' driver = webdriver.firefox() driver.get(url)  soup = beautifulsoup(driver.page_source)  x in soup.findall('li', {'class':'photo'}):     print x

now soup should expecting.


Comments

Post a Comment

Popular posts from this blog

matlab - Deleting rows with specific rules -

i got 20*3 cell array , need delete rows contains "137", "2" , "n:t" origin data: 't' '' '' 'np(*)' '' '' [ 137] '' '' [ 2] '' '' 'are' 'and' 'np(fcc_a1#1)' '' '' '1:t' [ 1200] [0.7052] '' [1.2051e+03] [0.7076] '' 'are' 'and' 'np(fcc_a1#3)' '' '' '2:t' [ 1200] [0.0673] '' [1.2051e+03] [0.0671] '' 'are' 'and' 'np(m23c6)' '' '' '3:t' [ 1200] [0.227...
Read more

php - MySQLi multi_query results for later use -

i using mysqli multi_query work several select statemets @ time. know how handle results, able use them later in code. example: <?php //connection stuff $query = "select name, surname database1;"; $query.= "select car, year, type database2 carid='1';"; $query.= "select product, price database3;"; if ($mysqli->multi_query($query)) { if($result = $mysqli->store_result()) { while($row = $result->fetch_row()) { --> here? } } } ?> <html> <div id='persona'> <?php foreach() { --> print name + surname } ?> </div> <div id='cars'> <?php foreach() { --> print car + year + type } ?> </div> <div id='product'> <?php foreach() { --> print product + price } ?> </div> </html> one more thing, prepared statements not possible when using multiple_query, right? there no benefit in putting unrelated queries in 1 multi ...
Read more