Question

4

Errors attempting to run GPT detector under Ubuntu 22.04 WSL

rated 0 times [ 4] [ 0] / answers: 1 / hits: 1696 / 2 Years ago, sun, april 10, 2022, 7:26:29

While the OpenAI detector has been useful in identifying content that is created by ChatGPT and other OpenAI-based models, as usage increases (especially by users here on Stack Exchange sites), it's been down more and more frequently.

After installing it locally per the project README, I receive the following error when attempting to run it from the repo directory using python -m detector.server ../gpt-2-models/detector-base.pt:

Traceback (most recent call last):

  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main

    return _run_code(code, main_globals, None,

  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code

    exec(code, run_globals)

  File "/home/ntd/src/gpt-2-output-dataset/detector/server.py", line 120, in <module>

    fire.Fire(main)

  File "/home/ntd/src/venv/openai-detector/lib/python3.10/site-packages/fire/core.py", line 141, in Fire

    component_trace = _Fire(component, args, parsed_flag_args, context, name)

  File "/home/ntd/src/venv/openai-detector/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire

    component, remaining_args = _CallAndUpdateTrace(

  File "/home/ntd/src/venv/openai-detector/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace

    component = fn(*varargs, **kwargs)

  File "/home/ntd/src/gpt-2-output-dataset/detector/server.py", line 89, in main

    model.load_state_dict(data['model_state_dict'])

  File "/home/ntd/src/venv/openai-detector/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict

    raise RuntimeError('Error(s) in loading state_dict for {}:

	{}'.format(

RuntimeError: Error(s) in loading state_dict for RobertaForSequenceClassification:

        Missing key(s) in state_dict: "roberta.embeddings.position_ids".

        Unexpected key(s) in state_dict: "roberta.pooler.dense.weight", "roberta.pooler.dense.bias".

I attempted to change transformers==2.9.1 per comments in this issue, but then pip install -r requirements.txt fails as well.

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

brellked

Add To Favorites

Follow

Total Points: 63

Total Questions: 107

Total Answers: 104

Location: Morocco

Member since Fri, May 22, 2020

4 Years ago

brellked questions

1 RTL8821CE wifi fail after upgrading from 5.15.0-60 to 5.19.0-32

Wed, Jul 20, 22, 23:30, 2 Years ago

1 Grep ignore array of patterns

Mon, Sep 6, 21, 05:31, 3 Years ago

1 Fullscreen System monitor 20.04

Sat, Aug 20, 22, 02:03, 2 Years ago

1 Why yarn key update failes all the time? GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures couldn't be verified

Sun, Aug 21, 22, 07:11, 2 Years ago

1 Should I install Ubuntu on and old HP laptop (HP 250 G3)?

Fri, May 28, 21, 21:12, 3 Years ago

View All

answered 2 Years ago terneive · Accepted Answer

And one more solution, in case you can't use my Docker-based answer for some reason. I believe this should replace my original solution as the preferred method.

Using the Dockerfile for the HuggingFace space as a guide, I've been able to reproduce this on a fresh Ubuntu 22.04. I'm running it on WSL, but thanks to @cocomac for confirming this also works on stock Debian. It should run on any recent Debian-based distribution:

sudo apt update && sudo apt upgrade -y



sudo apt install -y 

    git 

    make build-essential libssl-dev zlib1g-dev 

    libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm 

    libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev git-lfs  

    ffmpeg libsm6 libxext6 cmake libgl1-mesa-glx



git lfs install

cd ~

git clone https://huggingface.co/spaces/openai/openai-detector

cd openai-detector



PATH=/home/user/.local/bin:$PATH

curl https://pyenv.run | bash

source ~/.bashrc

PATH=$HOME/.pyenv/shims:$HOME/.pyenv/bin:$PATH



pyenv install 3.7.5

# ^ Takes a while



pyenv global 3.7.5

pyenv rehash



python --version

# Confirm 3.7.5



pip install --no-cache-dir --upgrade pip setuptools wheel

pip install --no-cache-dir 

    datasets 

    huggingface-hub "protobuf<4" "click<8.1"



pip install --no-cache-dir -r requirements.txt

# Ignore dependency errors, as it still appears to work



python -m detector.server detector-base.pt --port 7860

For future launches, add the following to something like a webui.sh script:

PATH=$HOME/.pyenv/shims:$HOME/.pyenv/bin:$PATH

pyenv global 3.7.5

pyenv rehash

cd ~/openai-detector

python -m detector.server detector-base.pt --port 7860