External Learning & Competitions

[2024][Basic][P2] Real and Fake Job Postings

I revisited the interrupted BERT run and built a duplicate-aware TF-IDF and LinearSVC baseline.

Platform: Kaggle
Period: 2024-08-11–2024-08-27
Host: sw.baek_00
Metric: Accuracy
Model rule: Use the pretrained BERT model; other models are not allowed

Kaggle P2 Real and Fake Job Postings overview — The saved screen confirms the BERT-only rule and Accuracy metric.

Where the earlier run stopped

The notebook loaded 12,516 training rows and 5,364 test rows, but training stopped at step 51 of 352. I treated the preprocessing and partial output as a historical run, not a completed BERT result.

Duplicate-aware split

Exact duplicate descriptions stay in one group before the train, calibration, and holdout split. This prevents a model from seeing the same job description on both sides.

Local baseline

Word and character TF-IDF with a class-weighted LinearSVC reached 0.9896 accuracy, 0.9307 balanced accuracy, 0.8879 F1, and 0.9187 average precision on the holdout. I reported minority-class precision and recall alongside accuracy.

The old notebook contained a plaintext API key. I excluded it, moved credential access to an environment variable, and left revocation and replacement to the credential owner. These numbers are local holdout results, not Kaggle leaderboard results.

What I changed for the rerun

I compared the original notebook result with a local holdout run. I used word+character TF-IDF + class-weighted LinearSVC; threshold selected on calibration F1 under description-grouped stratified train/calibration/holdout; 8,009/2,003/2,504 rows; seed 42.

Earlier run: train 12,516 rows; test 5,364 rows; training stopped at 51/352 steps (14%)
Simple baseline: accuracy 0.9525; balanced accuracy 0.5000; F1 0.0000; recall 0.0000
Reworked model: accuracy 0.9896; balanced accuracy 0.9307; F1 0.8879; precision 0.9115; recall 0.8655; AP 0.9187

Exact duplicate descriptions stay in one split. The sanitized rerun contains no API key, but the owner must revoke and replace the credential exposed in the original notebook. Kaggle upload and leaderboard evaluation remain unverified.

The old notebook contained a plaintext API key. I excluded that file and changed the public example to read credentials from an environment variable. The old key still needs to be revoked and replaced by its owner.

Open the rerun scripts and sanitized notebooks

These numbers come from my documented local split, not from the Kaggle leaderboard. I did not reproduce a signed-in submission history or rank.