running NER fails (in my venv)

[skurzinz]

tei-publisher-ner is running in a venv env in the folder I cloned this repo to (maybe related to #5).

Training on a set of TEIs directly from TEIPublisher worked, and a model is indeed created in a models folder: amazing!

Yet when -- after a restart of the python server -- trying to run NER from the annotation playground in TEIPublisher, nothing happens (no popup to choose a model from), and there is a 404 thrown by http://localhost:8080/exist/apps/tei-publisher/api/nlp/status/models: {"code":"errors:NOT_FOUND_404","value":null,"module":"/db/apps/tei-publisher/modules/lib/api/nlp.xql","line":39,"column":13,"description":"An error has been raised by the query [at line 39 column 13 in module /db/apps/tei-publisher/modules/lib/api/nlp.xql]"}

I’m pasting the output from the terminal running the server below, hope this is enough info for some debugging (full absolute path redacted).

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/fastapi/routing.py", line 255, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tei-publisher-ner/env/lib64/python3.11/site-packages/fastapi/routing.py", line 141, in serialize_response
    raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: <exception str() failed>

[HugoSchtr]

Hello! I have a similar error after adding a custom NER model to the models folder. @skurzinz did you manage to bypass the error?

[skurzinz]

@skurzinz did you manage to bypass the error?

Nope, I gave up as there was no further response here. Let me know if you succeed! :)

[HugoSchtr]

Thanks for your answer! I'll keep you updated if I find a way. 👍

[HugoSchtr]

I have tried modifying main.py, but with no success.

I have tried using a SpaCy NER model directly trained with TEI Publisher GUI, and I also tried using a custom SpaCy transformer NER model, trained with a newer SpaCy version environment (so outside TEI Publisher GUI).

I'm copy-pasting the error traceback here for referencing it in the future.

In the traceback, TEI Publisher SpaCy NER model is referred as modelTest, and custom SpaCy transformer NER model as modelTransformer.

Traceback (most recent call last):
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/fastapi/routing.py", line 292, in app
    content = await serialize_response(
  File "/Dev/tei-publisher-ner/venv/lib/python3.8/site-packages/fastapi/routing.py", line 155, in serialize_response
    raise ResponseValidationError(
fastapi.exceptions.ResponseValidationError: 2 validation errors:
  {'type': 'string_type', 'loc': ('response', 0), 'msg': 'Input should be a valid string', 'input': PosixPath('modelTransformer'), 'url': 'https://errors.pydantic.dev/2.4/v/string_type'}
  {'type': 'string_type', 'loc': ('response', 1), 'msg': 'Input should be a valid string', 'input': PosixPath('modelTest'), 'url': 'https://errors.pydantic.dev/2.4/v/string_type'}