Skip to content

feat: KBO 크롤링 HTTP 호출 전환#1050

Open
jjunh0 wants to merge 3 commits intodevfrom
feat/1049
Open

feat: KBO 크롤링 HTTP 호출 전환#1050
jjunh0 wants to merge 3 commits intodevfrom
feat/1049

Conversation

@jjunh0
Copy link
Copy Markdown
Contributor

@jjunh0 jjunh0 commented May 3, 2026

Summary

  • GameCenter 크롤러를 Playwright 렌더링 대신 허용된 GameCenter HTML GET + Jsoup 파싱 기반으로 전환했습니다.
  • Review 크롤러를 Playwright 렌더링 대신 허용된 Review HTML GET + Jsoup 파싱 기반으로 전환했습니다.
  • Scoreboard 크롤러를 Playwright 날짜 선택 대신 ScoreBoard.aspx GET 후 ASP.NET AJAX POST + Jsoup .smsScore 파싱 기반으로 전환했습니다.
  • robots.txt에서 Disallow된 /ws/ 호출을 제거했고, crawling 모듈의 Playwright 의존성 및 잔여 page/manager 클래스를 제거했습니다.
  • HTTP postback fragment 파서와 Scoreboard Jsoup 파서 테스트를 추가/갱신했습니다.

Verification

  • ./gradlew :crawling:test --no-daemon

Note

  • Scoreboard는 /Schedule/ScoreBoard.aspx에 hidden form field, cookie, hfSearchDate를 포함해 POST하는 방식이라 /ws/를 호출하지 않습니다.
  • 현재 KBO raw Main.aspx/Review HTML은 실제 경기 목록과 리뷰 기록 row를 서버 렌더링하지 않고 빈 컨테이너 및 스크립트만 내려주는 케이스가 확인되었습니다. 이 구현은 /ws/를 호출하지 않기 때문에 robots.txt는 준수하지만, KBO가 HTML에 데이터를 포함하지 않는 날짜/페이지에서는 빈 결과가 반환됩니다.

API Spec

  • 애플리케이션 외부 API 추가/변경 없음.

Resolves #1049

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 3, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 347f6edf-58e5-4bed-82be-a3298633b337

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/1049

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added 🐒 BE 백엔드 🐒 🚀 feat 기능 (새로운 기능, 기능 수정) labels May 3, 2026
@bowook
Copy link
Copy Markdown
Member

bowook commented May 6, 2026

야구보구를 돌려내!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐒 BE 백엔드 🐒 🚀 feat 기능 (새로운 기능, 기능 수정)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants