请教，怎么爬天气网的 json 数据

@zhihaofans
也设了，chrome 里正常访问的 request header 我都给弄了进去，还是不行

｛'Host': 'd1.weather.com.cn',
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
'Accept': '*/*'',
'Referer': 'http://m.weather.com.cn/mweather/101020100.shtml',
'Accept-Encoding': 'gzip, deflate, sdch',
'Accept-Language': 'zh-CN,zh;q=0.8',
'Cookie': 'BIGipServerd1src_pool=1874396221.20480.0000'｝

lxml

2017-08-10 18:13:39 +08:00

```
import requests

cookies = {
# 自己填写
}

headers = {
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.8',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36',
'Accept': '*/*',
'Referer': 'http://m.weather.com.cn/maqi/101020100.shtml',
'Connection': 'keep-alive',
}

params = (
('_', '14483821791309'),
)

a = requests.get('http://d1.weather.com.cn/aqi_all/101020100.html', headers=headers, params=params, cookies=cookies)
a.encoding = 'utf-8'
print(a.text)
```
刚刚用 curl 生成了一份代码，实测是可以的，你可能是没带 cookie ？

q409195961

2017-08-10 18:16:14 +08:00

带上 Referer: http://m.weather.com.cn/maqi/101020100.shtml
就不会 403

visitantzj

2017-08-10 18:24:56 +08:00

@lxml 感谢老兄！

看来应该是 cookie 的问题，我这写法里 urllib.request.Request 应该是没把 cookie 传递出去。非常感谢！知道大方向自己就好去 google 了

url = r"http://d1.weather.com.cn/aqi_7d/XiangJiAqiFc5d/101020100.html"
headers = {
'Host': 'd1.weather.com.cn',
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
'Accept': '*/*',
'Referer': 'http://m.weather.com.cn/maqi/101020100.shtml',
'Accept-Encoding': 'gzip, deflate, sdch',
'Accept-Language': 'zh-CN,zh;q=0.8',
'Cookie': 'f_city=%E9%95%87%E6%B1%9F%7C101190301%7C; BIGipServerd1src_pool=1874396221.20480.0000'
}
req = urllib.request.Request(url= url, headers=headers)

resp = urllib.request.urlopen(url)
respBytes = resp.read()
html = respBytes.decode('utf-8')

visitantzj

2017-08-10 18:26:38 +08:00

@q409195961 不好意思，这个是开始试另一个页面时候的 url 没改过来，我感觉应该是像楼上说的是 cookie 问题……自己先研究一下先

q409195961

2017-08-10 18:33:22 +08:00

@visitantzj 如图
![image]( https://i.loli.net/2017/08/10/598c3642cffd6.png)
[img]https://i.loli.net/2017/08/10/598c3642cffd6.png[/img]
<img src="https://i.loli.net/2017/08/10/598c3642cffd6.png"/>

(PS：试一下哪种方式可以插图)