celeo / rfesi Goto Github PK

Having rfesi log errors results in a lot of unnecessary messages, when I'm handling them in the code that's calling rfesi.

For example: rfesi logs every 404, but intentionally encountering 404s is the only way to make sure that every page of data is retrieved until the response headers (#34) are exposed to end users.

    #[derive(Debug)]
    pub enum EveClientError {
        Esi(EsiError),
        Other(anyhow::Error),
    }

    impl From<EsiError> for EveClientError {
        fn from(value: EsiError) -> Self {
            Self::Esi(value)
        }
    }

    impl From<anyhow::Error> for EveClientError {
        fn from(value: anyhow::Error) -> Self {
            Self::Other(value)
        }
    }

    impl From<EveClientError> for anyhow::Error {
        fn from(value: EveClientError) -> Self {
            match value {
                EveClientError::Esi(esi_error) => anyhow!(esi_error),
                EveClientError::Other(other) => other,
            }
        }
    }

    pub async fn fetch_all_pages<T, L>(mut fetch_page: L) -> Result<Vec<T>>
    where
        L: AsyncFnMut<(i32,), Output = std::result::Result<Vec<T>,EveClientError>>
    {
        let mut results = Vec::<T>::new();
        let mut current_page = 1;

        loop {
            match fetch_page.call_mut((current_page,)).await {
                Ok(page_results) => results.extend(page_results),
                Err(EveClientError::Esi(EsiError::InvalidStatusCode(404))) => break,
                Err(e) => return Err(e.into()),
            }
            current_page += 1;
        }

        Ok(results)
    }

        let mut esi = EsiBuilder::new()
            .user_agent("Your user agent")
            .build()?;
        esi.update_spec().await?;

        let new_market_data = crate::eve_client::fetch_all_pages(
            async_owned_closure_mut!(
                {
                    esi: Esi = esi,
                    region: i32 = region,
                };
                async |page_no: i32| -> std::result::Result<Vec<MarketOrder>, EveClientError> {
                    debug!("Fetching market orders for region {region} page {page_no}.");
                    esi.group_market().get_region_orders(*region, None, Some(page_no), None).await.map_err(Into::into)
                }
            )
        ).await?;

It seems that having rfesi return a Result (as it already does) should be sufficient, and it shouldn't need to also log at the error level internally, especially when it's an error that is able to be handled by the user of the library.

For the app I'm building, I'm exclusively querying public APIs, and therefore haven't gone through the trouble of setting up an app on EVE. What are your thoughts on supporting this use case? Currently EsiBuilder will panic if its missing any of client id, client secret, or callback URL, all of which are unnecessary if only using the public API.

EsiBuilder could potentially not panic if it's missing those three values and produce a "public only" Esi struct which panics on any Api call not marked with RequestType::Public instead.

https://docs.esi.evetech.net/docs/esi_introduction.html#rules states that we should be looking at the X-ESI-Error-Limit-Remain, X-ESI-Error-Limit-Reset, and expires headers in the response, but I don't see a way to access them using rfesi. A similar issue was brought up about the x-pages header in #6.

Trying to run rfesi I'm getting 'Error constructing HTTP client' from using esi.group_character().get_public_info(player.id).await?;. What Can I do because I'm authenticating and using other functions like esi.group_character().get_portrait(player.id).await?;without any problems:

https://github.com/rafaga/webb/blob/0b93d62b8f94326633d1b7fc405b0b56b49b25bb/src/esi.rs#LL206C10-L206C10

   #[tokio::main]
   pub async fn auth_user(&mut self,port: u16) -> Result<Option<Character>, Box<dyn std::error::Error + Send + Sync>> {
        [...]
        let claims = self.esi.authenticate(result.0.as_str()).await?;      

        let mut player = Character::new();  
        let data = claims.unwrap();
        player.name = data.name;
        let split:Vec<&str> = data.sub.split(":").collect();
        player.id = split[2].parse::<u64>().unwrap();
        if player.auth != None {
            [...]
            //IF I OMIT THIS LINE AN EXCEPTION ARISE
            self.esi.update_spec().await?;
            
            //THIS LINE RAISE THE EXCEPTION  'Error constructing HTTP client'
            let public_info = self.esi.group_character().get_public_info(player.id).await?;
            [...]

            //THIS LINE RUNS FINE
            let player_portraits = self.esi.group_character().get_portrait(player.id).await?;
            player.photo = Some(player_portraits.px64x64);
        }
        Ok(Some(player))
    }

I can't determine why there is an error here. please help me if you are not so busy.

running 1 test
2023-05-29 20:15:12.338628-0600 database-98a788513e93f095[34694:460942] SecTaskLoadEntitlements failed error=22 cs_flags=20, pid=34694
2023-05-29 20:15:12.339286-0600 database-98a788513e93f095[34694:460942] SecTaskCopyDebugDescription: database-98a7885[34694]/0#-1 LF=0
thread 'tests_database::test_add_character' panicked at 'An error occurred has ocurred: Error constructing HTTP client', tests/database.rs:90:25
stack backtrace:
   0: rust_begin_unwind
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579:5
   1: core::panicking::panic_fmt
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:64:14
   2: database::tests_database::test_add_character
             at ./tests/database.rs:90:25
   3: database::tests_database::test_add_character::{{closure}}
             at ./tests/database.rs:65:29
   4: core::ops::function::FnOnce::call_once
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ops/function.rs:250:5
   5: core::ops::function::FnOnce::call_once
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
test tests_database::test_add_character ... FAILED

failures:

failures:
    tests_database::test_add_character

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 3 filtered out; finished in 28.51s

When no scopes are requested the returned jwt from the Esi does not contain the scp field and serde throws an error when its decoded.

Hi, I'd like to add support for the /universe/ids POST endpoint.

I am working on a Rust application and need to support a way to convert character names into character IDs, and this endpoint seemed the correct way.

Is this currently being worked on, or is it something I can tackle and open a PR for?

Thank you!

The include_completed parameter on the character industry jobs ESI endpoint is not available through the IndustryGroup::get_character_industry_jobs interface.

If the struct instance has a refresh token specified, it'd be convenient if it used that token automatically when the access token expired to fetch another.

I had to make a change in my fork to be able to connect, otherwise I was getting 404 errors (trying the assets endpoint). The change I made was in client.rs to use the BASE_URL for all the API endpoints, rather than using the OAUTH_URL when authenticated.

Just wondering if something's changed, or if I'm misusing it?

Hi,

One challenge I have is that I'm storing a bunch of accounts, and then I loop over them, creating an ESI object for each, updating, then discarding. But it's really slow to call update_spec on each of those each time.

I had thought I could cache the 'spec' value for the life of the program, but the spec value is private so I can't manually specify it.

Do you have any suggestions on how to resolve this? I wondered about adding a method to allow manual setting of spec (or making it public), or perhaps some other approach with the existing code that I'm not thinking of?

Perhaps I should be creating and storing the ESI object itself as a whole once, and then cloning that each time I need it (and updating the access/refresh/expiration values). Scopes don't differ, so that might work.

Thanks!

like I was mentioning on #13 I can't add any new users using ESI because audience field in the claims aren't validating.

I want to know if you can reproduce the problem, how to do it:

• Authenticate a new user into ESI using this crate.

Throws an error "JWT audience field is incorrect" on v0.19.0. Such field comes with None value.

One thing I've been noticing as I've been going through various endpoints is that rfesi uses u32/u64 in a number of places where the ESI docs say it uses an integer32/integer64 (most commonly with IDs). There's also that rfesi uses u128 for the access token expiration (millisecond timestamp), but chrono uses i64 (DateTime timestamp_millis). It is very unlikely that CCP will give an access token that expires before 1970, but it does make it a bit awkward to work with the expiration.

While it's very unlikely that CCP will be returning negative numbers in a lot of these places, they do only document them as "integer", instead of as "unsigned integer". I've been hitting this with IDs, because I'm using PostgreSQL, and it doesn't support unsigned integers without a 3rd party extension (mostly because they're not part of the SQL standard). Because PostgreSQL & the SQL standard don't support unsigned ints, sqlx also doesn't support them.

Mostly I wanted to bring these up as awkward points of integration with other things, and see how you've been handling them, and what your thoughts where on the reasons behind the specific type choices.

I don't really understand how to use the structs in the groups module, and I couldn't find any sample code in the documentation. For example, I was trying to get all the market orders in a region so I started by looking at the documentation for Esi and found the group_market function with the description of "Call endpoints under the "Market" group in ESI.". So far so good. I then saw it returned a MarketGroup struct, so I went to see what I can do with that here. But I couldn't find anything useful in that documentation, like a function to do the actual endpoint querying.

Adding support for the System list and Facilities list from the Industry endpoitn