[Python] 파이썬 크롤링 urllib.error.HTTPError: HTTP Error 403: Forbidden 오류 처리 방법

Request 사용시 오류가 발생하였습니다.

오류내용은 urllib.error.HTTPError: HTTP Error 403: Forbidden 입니다.

  File "C:pythonWorkspacemain.py", line 59, in <module>
    html = urlopen(request).read()
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 523, in open
    response = meth(req, response)
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 632, in http_response
    response = self.parent.error(
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 561, in error
    return self._call_chain(*args)
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 494, in _call_chain
    result = func(*args)
  File "C:UsersilikeAppDataLocalProgramsPythonPython39liburllibrequest.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

오류가 발생한 코드는 다음과 같습니다.

import urllib.request
import urllib.parse
from bs4 import BeautifulSoup

url = '타켓URL'

request = urllib.request.urlopen(url)
html = request.read()
beautifulSoup = BeautifulSoup(html, 'html.parser')
............생략

urlopen메서드를 사용하여 url 여는 과정에서 발생되었습니다. 

[해결방법]

Request()메서드 호출시 header 정보를 추가하여 시도해보세요.

from urllib.request import Request, urlopen

req = Request('타켓팅 URL', headers={'User-Agent': 'Mozilla/5.0'})
webpage = urlopen(req).read()

[REFERENCE]

stackoverflow.com/questions/16627227/http-error-403-in-python-3-web-scraping