I get this error in PAWS on v0.3.3 when doing single subject:

Picking a random main subject
Working on naturgas
Do you want to continue? [Y/Enter/n]: 
Traceback (most recent call last):
  File "/home/paws/.itemsubjector/itemsubjector.py", line 8, in <module>
    itemsubjector.run()
  File "/home/paws/.itemsubjector/src/__init__.py", line 79, in run
    main_subjects.get_validated_main_subjects_as_jobs()
  File "/home/paws/.itemsubjector/src/models/main_subjects.py", line 108, in get_validated_main_subjects_as_jobs
    job = main_subject_item.fetch_items_and_get_job_if_confirmed()
  File "/home/paws/.itemsubjector/src/models/wikimedia/wikidata/item/main_subject.py", line 240, in fetch_items_and_get_job_if_confirmed
    return self.__fetch_and_parse__()
  File "/home/paws/.itemsubjector/src/models/wikimedia/wikidata/item/main_subject.py", line 250, in __fetch_and_parse__
    self.__prepare_before_fetching_items__()
  File "/home/paws/.itemsubjector/src/models/wikimedia/wikidata/item/main_subject.py", line 188, in __prepare_before_fetching_items__
    self.__extract_search_strings__()
  File "/home/paws/.itemsubjector/src/models/wikimedia/wikidata/item/main_subject.py", line 141, in __extract_search_strings__
    elif self.id in config.no_alias_for_scholarly_items:
AttributeError: module 'config' has no attribute 'no_alias_for_scholarly_items'

My command was poetry run python itemsubjector.py -a Q40858

On checking the latest version and 0.3-alpha2, I am getting the following error:

Traceback (most recent call last):
  File "itemsubjector.py", line 3, in <module>
    import src
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/tmp/ItemSubjector-0.3-alpha2/src/__init__.py", line 11, in <module>
    from src.helpers.console import (
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/tmp/ItemSubjector-0.3-alpha2/src/helpers/console.py", line 11, in <module>
    from src.models.batch_job import BatchJob
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/tmp/ItemSubjector-0.3-alpha2/src/models/batch_job.py", line 3, in <module>
    from src.models.items import Items
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/tmp/ItemSubjector-0.3-alpha2/src/models/items/__init__.py", line 10, in <module>
    from src.models.wikimedia.wikidata.sparql_item import SparqlItem
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/tmp/ItemSubjector-0.3-alpha2/src/models/wikimedia/wikidata/sparql_item.py", line 4, in <module>
    import config.items
ModuleNotFoundError: No module named 'config.items'

After installing in PAWS I ran this command:

poetry run python itemsubjector.py -a Q40858

I then selected 2 to work on Riksdagen documents. This caused this screen:

Working on naturgas, see http://www.wikidata.org/entity/Q40858
Got a total of 78 items
Please keep an eye on the lag of the WDQS cluster here and avoid working if it is over a few minutes.
https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&viewPanel=8&from=now-30m&to=now&refresh=1d You can see if any lagging servers are pooled here
https://config-master.wikimedia.org/pybal/eqiad/wdqs
If any enabled servers are lagging more than 5-10 minutes you can search phabricator for open tickets to see if the team is on it.
If you don't find any feel free to create a new ticket like this:
https://phabricator.wikimedia.org/T291621
Running 1 job(s) with a total of 1 items non-interactively now. You can take a coffee break and lean back :)
Traceback (most recent call last):
  File "/home/paws/.itemsubjector/itemsubjector.py", line 8, in <module>
    itemsubjector.run()
  File "/home/paws/.itemsubjector/src/__init__.py", line 164, in run
    handle_job_preparation_or_run_directly_if_any_jobs(
  File "/home/paws/.itemsubjector/src/helpers/jobs.py", line 154, in handle_job_preparation_or_run_directly_if_any_jobs
    batchjobs.run_jobs()
  File "/home/paws/.itemsubjector/src/models/batch_jobs.py", line 45, in run_jobs
    job.suggestion.add_to_items(
  File "/home/paws/.itemsubjector/src/models/suggestion.py", line 111, in add_to_items
    f"to {clean_rich_formatting(target_item.label)}"
  File "/home/paws/.itemsubjector/src/helpers/cleaning.py", line 24, in clean_rich_formatting
    return label.replace("[/", "['/")
AttributeError: 'NoneType' object has no attribute 'replace'

Labels with apostrophe(') currently do not work. I think that an escape character needs to be added, before sending the query string to WDQS.

Take for example:

Alzheimer's disease (Q11081)

returns the following error

Fetching items with labels that have one of the search strings by running a total of 11 queries on WDQS...INFO:backoff:Backing off execute_sparql_query(...) for 1.0s (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url

In v0.2 I am trying pip install -r requirements.txt in PAWS and get this error message:

Collecting wikibaseintegrator
  Cloning git://github.com/LeMyst/WikibaseIntegrator (to revision v0.12.0.dev5) to /tmp/pip-install-h0jhod33/wikibaseintegrator_2f94ad8cb5b244b3816e997a960745eb
  Running command git clone --filter=blob:none --quiet git://github.com/LeMyst/WikibaseIntegrator /tmp/pip-install-h0jhod33/wikibaseintegrator_2f94ad8cb5b244b3816e997a960745eb
  fatal: unable to connect to github.com:
  github.com[0: 140.82.113.4]: errno=Connection timed out

  error: subprocess-exited-with-error
  
  × git clone --filter=blob:none --quiet git://github.com/LeMyst/WikibaseIntegrator /tmp/pip-install-h0jhod33/wikibaseintegrator_2f94ad8cb5b244b3816e997a960745eb did not run successfully.
  │ exit code: 128
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet git://github.com/LeMyst/WikibaseIntegrator /tmp/pip-install-h0jhod33/wikibaseintegrator_2f94ad8cb5b244b3816e997a960745eb did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

What should I do?

Thanks for correcting the previous errors in 0.3-alpha3.

I checked out the latest commit in the main branch and 0.3-alpha4. And now, I now face the OAuth error.

1. I checked with username/password
2. I checked with botname/password

  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/my_venv/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 432, in validate_token_parameters
    raise_from_error(params.get('error'), params)
  File "/mnt/nfs/labstore-secondary-tools-project/itemsubjector-jsamwrites/itemsubjector/my_venv/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 402, in raise_from_error
    raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidClientIdError: (invalid_request) The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed.

Any idea on this error.

I checked with other scripts of mine. There are no issues.

I want to add the following main subjects to the articles

1. scoping review protocol (Q108684373): very specific
2. scoping review (Q101116078): generic

and I run the following command (from specific topics to generic topics)

$ python itemsubjector.py -na -l Q108684373 Q101116078

Even though the addition of 'Q108684373' is complete, I see articles with the text 'scoping review protocol' in the list for 'scoping review'.

This issue may be related to Issue 14

This pull request implements asking in the end when --limit is used. Hopefully that improves the UX.

Also pydantics BaseModel is now used for validation.

The QS export support was dropped to reduce complexity. Also adding main subjects already present on articles was dropped.

When working on HCV https://www.wikidata.org/wiki/Q154869 I see that there is another item with "HCV 229E" and I want to make sure that they don't get included in my batch.

This is useful, because there are already many thousand different main subjects and many of them are not matched properly with all relevant articles yet.

As a user, I want to choose whether to search for items matching the label and or one of the aliases so I get as many hits as possible.

Pseudo code: also fetch the aliases from WDQS ask user for which ones to include (or all) https://console-menu.readthedocs.io/en/latest/consolemenu/MultiSelectMenu.html add them to a new attribute in class Labels: search_strings fetch based on that (with one query if possible) use https://pmitzias.com/SPARQLBurger/docs.html to generate the SPARQL query using UNION

Bumps certifi from 2022.9.24 to 2022.12.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Similar to QuickStatements batches, ItemsSubjector could have a flask frontend that runs in Toolforge and execute the users batches.

This requires oauth and flask. Lucas made a good toolforge flask template to get started.