Scrapy 获取不到拉勾网指定的xpath数据

Scrapy 获取不到拉勾网指定的xpath数据-鸿蒙开发者社区

用scrapy shell调试,也没出来数据

Scrapy 获取不到拉勾网指定的xpath数据-鸿蒙开发者社区

爬虫小白,在线求大佬指点!

​settings.py​

BOT_NAME = 'lagou'


SPIDER_MODULES = ['lagou.spiders']

NEWSPIDER_MODULE = 'lagou.spiders'


#指定Log级别

LOG_LEVEL = 'ERROR'

#LOG_FILE = 'lagou.log'


# Crawl responsibly by identifying yourself (and your website) on the user-agent

USER_AGENT = [

    'MSIE (MSIE 6.0; X11; Linux; i686) Opera 7.23',

    'Opera/9.20 (Macintosh; Intel Mac OS X; U; en)',

    'Opera/9.0 (Macintosh; PPC Mac OS X; U; en)',

    'iTunes/9.0.3 (Macintosh; U; Intel Mac OS X 10_6_2; en-ca)',

    'Mozilla/4.76 [en_jp] (X11; U; SunOS 5.8 sun4u)',

    'iTunes/4.2 (Macintosh; U; PPC Mac OS X 10.2)',

    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0',

    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0',

    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20120813 Firefox/16.0',

    'Mozilla/4.77 [en] (X11; I; IRIX;64 6.5 IP30)',

    'Mozilla/4.8 [en] (X11; U; SunOS; 5.7 sun4u)'

]


PROXIES = [

    {

        'ip_port': '61.216.156.222:60808',

        'user_pass': ''

    },

    {

        'ip_port': '183.236.232.160:8080',

        'user_pass': ''

    },

    {

        'ip_port': '222.74.73.202:42055',

        'user_pass': ''

    },

    {

        'ip_port': '210.5.10.87:53281',

        'user_pass': ''

    },

    {

        'ip_port': '183.236.232.160:8080',

        'user_pass': ''

    },

    {

        'ip_port': '61.216.156.222:60808',

        'user_pass': ''

    },

]


# Obey robots.txt rules

ROBOTSTXT_OBEY = False


# Configure maximum concurrent requests performed by Scrapy (default: 16)

#CONCURRENT_REQUESTS = 32


# Configure a delay for requests for the same website (default: 0)

# See https://docs.scrapy.org/en/latest/topics/settings.html#download-delay

# See also autothrottle settings and docs

DOWNLOAD_DELAY = 3

# The download delay setting will honor only one of:

#CONCURRENT_REQUESTS_PER_DOMAIN = 16

#CONCURRENT_REQUESTS_PER_IP = 16


# Disable cookies (enabled by default)

COOKIES_ENABLED = False


Scrapy 获取不到拉勾网指定的xpath数据-鸿蒙开发者社区

用xpath插件数据没问题啊,哭了...

急急急,在线等大佬指点!

python 爬虫
2022-11-20 10:52:33
浏览
收藏 0
回答 0
待解决
相关问题
HarmonyOS图片压缩不到指定大小
478浏览 • 1回复 待解决
获取指定月份天数。
333浏览 • 1回复 待解决
native侧log获取不到
1555浏览 • 1回复 待解决
openharmony怎么获取以太MAC地址?
2493浏览 • 1回复 待解决
Preferences获取不到
8878浏览 • 2回复 待解决
用户相册, 获取不到albumName
1685浏览 • 1回复 待解决
如何获取指定Bundle NameAbility信息
1880浏览 • 1回复 待解决
鸿蒙应用开发请求不到数据
8044浏览 • 2回复 待解决
http request 请求不到接口数据
4811浏览 • 1回复 待解决
dataPreferences.Preferences取不到数据
129浏览 • 0回复 待解决
HarmonyOS如何获取指定子组件宽高
1087浏览 • 1回复 待解决
HarmonyOS 获取不到手机号
165浏览 • 1回复 待解决