Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 13 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,9 +263,8 @@ To upload to an organization by a name instead of its ID, save a name for it onc
The name is saved in the user profile in the config file:

```sh
# Upload to the organization, and save its ID under the name "work" in the profile "my_username"
mapillary_tools upload MY_CAPTURE_DIR \
--desc_path /tmp/mapillary_image_description.json \
# Save the ID of the organization under the name "work" in the profile "my_username"
mapillary_tools authenticate \
--user_name "my_username" \
--organization_key "my_organization_id" \
--organization_name "work"
Expand All @@ -277,6 +276,7 @@ mapillary_tools upload MY_CAPTURE_DIR \
--organization_name "work"
```

An upload can save the name too: give it `--organization_key` and `--organization_name` together.
`--organization_name` works with `upload`, `process_and_upload` and `video_process_and_upload`.
A name is 2 to 32 letters, numbers, hyphens and underscores, starting with a letter, and it is case-sensitive.
Saving a name that is already saved with another ID replaces the ID.
Expand Down Expand Up @@ -550,8 +550,16 @@ Authenticate for user `my_username`. If the user is already authenticated, it wi
mapillary_tools authenticate --user_name "my_username"
```

The command lists the existing profiles first, with the organization names saved in each of them
(see [Upload](#upload) to save one). Updating the credentials keeps the saved names.
The command lists the existing profiles first, with the organization names saved in each of them.
Updating the credentials keeps the saved names.

Save the ID of an organization under the name `work` in the profile `my_username`, to upload to it with
`--organization_name "work"` (see [Upload](#upload)). The profile is not logged in again, unless it is new or
`--jwt`, `--user_email` or `--user_password` is given:

```sh
mapillary_tools authenticate --user_name "my_username" --organization_key "my_organization_id" --organization_name "work"
```

Delete the organization name `work` saved in the profile `my_username`. The profile and its other saved names are kept:

Expand Down
14 changes: 9 additions & 5 deletions docs/command_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ For the full list of options of the version you have installed, run `mapillary_t
| `sample_video` | Extracts frames from videos as geotagged images. Needs FFmpeg. |
| `video_process` | `sample_video`, then `process` the frames. |
| `video_process_and_upload` | `video_process`, then `upload` the frames. |
| `authenticate` | Adds, updates or deletes user profiles, and deletes saved organization names. |
| `authenticate` | Adds, updates or deletes user profiles, and saves or deletes organization names. |
| `check_upload_history` | Tells whether processed files were already uploaded from this computer. |
| `zip` | Writes processed image sequences into zip files, to upload later. |

Expand Down Expand Up @@ -357,7 +357,7 @@ Uploads the images (as sequences), videos and zip files under the import paths,
mapillary_tools process MY_CAPTURE_DIR
mapillary_tools upload MY_CAPTURE_DIR --user_name my_profile

# Save the organization ID under the name "work" once, then use the name
# Save the organization ID under the name "work" once (or with authenticate), then use the name
mapillary_tools upload MY_CAPTURE_DIR --organization_key 1234567890 --organization_name work
mapillary_tools upload MY_CAPTURE_DIR --organization_name work
```
Expand Down Expand Up @@ -467,26 +467,30 @@ mapillary_tools authenticate [--user_name PROFILE] [OPTIONS]

Logs in to Mapillary and saves the credentials as a profile in the config file
(a file in `~/.config/mapillary/configs/`, or the file that `MAPILLARY_CONFIG_PATH` names). It lists the existing profiles first, with the
organization names saved in each.
organization names saved in each. It also saves and deletes the organization names of a profile.

| Option | Description |
| ---------------------------- | ----------------------------------------------------------------------------------- |
| `--user_name PROFILE` | The profile name: a local name of your choice, not necessarily your Mapillary user name. |
| `--user_email`, `--user_password` | Log in without prompts. |
| `--jwt TOKEN` | Save a user access token instead of logging in. |
| `--delete` | Delete the profile, or with `--organization_name NAME`, only that saved name. |
| `--organization_key ID` | With `--organization_name NAME`, save the organization ID under NAME in the profile. |
| `--organization_name NAME` | The organization name to save, or with `--delete`, to delete. |

```sh
mapillary_tools authenticate --user_name my_profile
mapillary_tools authenticate --user_name my_profile --delete
mapillary_tools authenticate --user_name my_profile --organization_key 1234567890 --organization_name work
mapillary_tools authenticate --user_name my_profile --organization_name work --delete
```

Gotchas:

- Authenticating an existing profile again replaces its credentials, and keeps its saved organization names.
- Organization names are saved by `upload`, not by `authenticate`
(`--organization_key ID --organization_name NAME`).
- Saving an organization name does not log an existing profile in again, unless `--jwt`, `--user_email` or
`--user_password` is given. It checks the profile's token and looks up the organization ID on Mapillary, so it
needs a connection. A new profile logs in first.
- Without a terminal, or with `MAPILLARY_TOOLS_PROMPT_DISABLED=YES`, give everything as options: missing values
are an error instead of a prompt.

Expand Down
114 changes: 104 additions & 10 deletions mapillary_tools/authenticate.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from __future__ import annotations

import contextlib
import getpass
import logging
import re
Expand All @@ -29,17 +30,41 @@ def authenticate(
user_password: str | None = None,
jwt: str | None = None,
delete: bool = False,
organization_key: str | None = None,
organization_name: str | None = None,
):
"""
Prompt for authentication information and save it to the config file
Prompt for authentication information and save it to the config file.

With organization_key and organization_name, save the organization ID under
the name in the profile, logging in only if the profile does not exist yet
or login options are given
"""

if organization_name is not None and not delete:
raise exceptions.MapillaryBadParameterError(
"authenticate takes --organization_name only with --delete. To save an "
"organization name, upload with --organization_key ID --organization_name NAME"
)
if organization_name is not None:
organization_name = organization_name.strip()
if organization_key is not None:
organization_key = organization_key.strip()

if delete:
if organization_key is not None:
raise exceptions.MapillaryBadParameterError(
"authenticate --delete takes --organization_name NAME, not --organization_key"
)
elif organization_key is None:
if organization_name is not None:
raise exceptions.MapillaryBadParameterError(
"authenticate takes --organization_name with --organization_key ID "
"to save the name, or with --delete to delete it"
)
else:
if organization_name is None:
raise exceptions.MapillaryBadParameterError(
"To save an organization, give the name to save its ID under: "
"--organization_key ID --organization_name NAME"
)
_validate_organization_name(organization_name)
_validate_organization_id(organization_key)

# We still have to accept --user_name for the back compatibility
profile_name = user_name
Expand All @@ -65,10 +90,33 @@ def authenticate(
assert profile_name is not None, "profile_name should be set"

if delete and organization_name is not None:
_delete_organization(profile_name, organization_name.strip(), all_user_items)
_delete_organization(profile_name, organization_name, all_user_items)
elif delete:
config.remove_config(profile_name)
LOG.info('Profile "%s" deleted', profile_name)
elif (
organization_key is not None
and profile_name in all_user_items
and not jwt
and user_email is None
and user_password is None
):
# Save the organization in the profile without logging in again, with
# the token saved in it checked
assert organization_name is not None
with _connection_errors_raised_as_mapillary_errors():
try:
profile_items = _verify_user_auth(
_validate_profile(all_user_items[profile_name])
)
except exceptions.MapillaryUploadUnauthorizedError as ex:
raise exceptions.MapillaryUploadUnauthorizedError(
f'{str(ex).rstrip(".")}. To log in to profile "{profile_name}" '
"again and save the organization, add --jwt TOKEN or --user_email EMAIL"
) from ex
_fetch_and_save_organization(
profile_name, profile_items, organization_key, organization_name
)
else:
if profile_name in all_user_items:
LOG.warning(
Expand Down Expand Up @@ -100,6 +148,47 @@ def authenticate(
'Profile "%s" created: %s', profile_name, http._sanitize(user_items)
)

if organization_key is not None:
assert organization_name is not None
_fetch_and_save_organization(
profile_name, user_items, organization_key, organization_name
)


def _fetch_and_save_organization(
profile_name: str,
user_items: config.UserItem,
organization_key: str,
organization_name: str,
) -> None:
with _connection_errors_raised_as_mapillary_errors():
data = _fetch_organization(user_items, organization_key)
saved_user_items = config.load_user(profile_name)
_save_organization(
profile_name,
organization_name,
# The ID as the server returns it
str(data.get("id") or organization_key),
data.get("name"),
config.list_organizations(saved_user_items) if saved_user_items else {},
)


@contextlib.contextmanager
def _connection_errors_raised_as_mapillary_errors() -> T.Iterator[None]:
try:
yield
except requests.Timeout as ex:
raise exceptions.MapillaryUploadTimeoutError(str(ex)) from ex
except requests.ConnectionError as ex:
raise exceptions.MapillaryUploadConnectionError(str(ex)) from ex


def _fetch_organization(user_items: config.UserItem, organization_id: str) -> T.Any:
with api_v4.create_user_session(user_items["user_upload_token"]) as session:
resp = api_v4.fetch_organization(session, organization_id)
return api_v4.jsonify_response(resp)


def fetch_user_items(
user_name: str | None = None,
Expand Down Expand Up @@ -189,9 +278,7 @@ def fetch_user_items(
)

if organization_id is not None:
with api_v4.create_user_session(user_items["user_upload_token"]) as session:
resp = api_v4.fetch_organization(session, organization_id)
data = api_v4.jsonify_response(resp)
data = _fetch_organization(user_items, organization_id)
LOG.info(
f"Uploading to organization: {data.get('name')} (ID: {data.get('id')})"
)
Expand Down Expand Up @@ -362,6 +449,13 @@ def _validate_organization_name(organization_name: str):
_validate_name(organization_name, "organization name")


def _validate_organization_id(organization_key: str):
if not _ORGANIZATION_ID_PATTERN.fullmatch(organization_key):
raise exceptions.MapillaryBadParameterError(
f'--organization_key takes an organization ID, a number, not "{organization_key}"'
)


def _list_all_profiles(profiles: dict[str, config.UserItem]) -> None:
_echo("Existing Mapillary profiles:")

Expand Down
8 changes: 7 additions & 1 deletion mapillary_tools/commands/authenticate.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,15 @@ def add_basic_arguments(self, parser: argparse.ArgumentParser):
required=False,
action="store_true",
)
parser.add_argument(
"--organization_key",
help="The Mapillary organization ID to save in the user profile under --organization_name NAME. An existing profile is not logged in again, unless login options are given.",
default=None,
required=False,
)
parser.add_argument(
"--organization_name",
help="The saved organization name to delete from the user profile, with --delete",
help="With --organization_key ID, save the ID under this name in the user profile, to upload with --organization_name NAME. With --delete, delete the saved name. Names are 2 to 32 letters, numbers, hyphens and underscores, starting with a letter.",
default=None,
required=False,
)
Expand Down
30 changes: 30 additions & 0 deletions tests/integration/test_organization_name.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,36 @@ def test_save_then_use_organization_name(
assert graph_api == [ORGANIZATION_ID, ORGANIZATION_ID]


def test_authenticate_saves_organization_name(
setup_config: py.path.local,
setup_data: py.path.local,
setup_upload: py.path.local,
graph_api: list[str],
):
# Saved without logging in again: the token stays
run_command(
[
*["--user_name", USERNAME],
*["--organization_key", ORGANIZATION_ID],
*["--organization_name", "work"],
],
command="authenticate",
)
assert config.load_user(USERNAME, config_path=str(setup_config)) == {
"user_upload_token": "test_user_token",
"MAPOrganization.work": ORGANIZATION_ID,
}
assert graph_api == [ORGANIZATION_ID]

run_process_and_upload_for_descs(
["--file_types=image", "--organization_name", "work"]
+ [str(setup_data.join("images"))]
)
keys = _history_organization_keys()
assert keys and set(keys) == {ORGANIZATION_ID}
assert graph_api == [ORGANIZATION_ID, ORGANIZATION_ID]


def test_video_process_and_upload_by_organization_name(
setup_config: py.path.local,
setup_data: py.path.local,
Expand Down
Loading
Loading