Fixes in bitpanda price fetching; more special case handling

* bitpanda general: Bitpanda Pro, whose API we use to fetch prices for bitpanda transactions, is now One Trading. As a result, their API is available under a different address and with a slightly different response. This commit contains the fixes for both, the address and the parsing/usage of the result.
* bitpanda general: Since One Trading doesn't offer the same coins as Bitpanda anymore, some coin/fiat pairs aren't available there (like BEST/EUR, ETHW/EUR, LTC/EUR and others). To differentiate between a "market" not being available and other errors, we raise a ValueError in case the "market" is not available. I chose ValueError because catching LookupError also catches errors with indices in lists, which should be thrown.
* bitpanda general: The ValueError is caught in price_data.py in the `get_cost` method. In the last commit, I added a new field `exported_price` to the `Operation` data class, which is used in the exception handling to use the price from the csv export (it's better than nothing). For BEST, there is sadly no price available if the BEST transaction is a Withdrawal (Fee). For now, we assume a value of 0 in that case (I don't know how to fix this otherwise).
* bitpanda LUNC airdrop: I added special handling for the LUNC airdrop that happened in May 2022 because of a blockchain crash and subsequent fork. Sadly, the price can't be retrieved using the API and the airdrop didn't have a price associated. CoinTaxman should throw an exception because it can't fetch a price and it should also say which line the airdrop is in. I used that to edit my csv export and input a ridiculously small price since the price was really low anyway.
* bitpanda staking rewards: Sometime before 2022/6/14, bitpanda used "transfer" for staking rewards. Incoming crypto "transfers" before that date, that aren't BEST, are therefore classified as (staking-)reward.

Author: wullxz

config.ini

@@ -28,7 +28,7 @@ LOG_LEVEL = DEBUG
 # If False, all airdrops will be taxed as `Einkünfte aus sonstigen Leistungen`.
 # Setting this config falsly will result in a wrong tax calculation.
 # Please inform yourself and help to resolve this issue by working on/with #115.
-# Some airdrops can be classified as gifts (Schenkung) or income (Eink├╝nfte)
+# Some airdrops can be classified as gifts (Schenkung) or income (Einkünfte)
 # relatively savely. For those, there is a flag to signal either type.
 # See the AirdropGift and AirdropIncome classes in transaction.py and their usage in book.py
 ALL_AIRDROPS_ARE_GIFTS = True

src/book.py

@@ -1099,7 +1099,7 @@ def _read_bitpanda(self, file_path: Path) -> None:
                 # CocaCola transfer, which I don't want to track. Would need to
                 # be implemented if need be.
                 if operation == "transfer":
-                    if asset == "BEST" and asset_class == "Cryptocurrency":
+                    if asset == "BEST" and asset_class == "Cryptocurrency" and inout == "incoming":
                         # BEST is awarded for trading activity and holding a portfolio at bitpanda
                         # The BEST awards are listed as "transfer" but must be processed as Airdrop (non-taxable)
                         operation = "airdrop_gift"
@@ -1132,6 +1132,48 @@ def _read_bitpanda(self, file_path: Path) -> None:
                             f"In row {row} in file {file_path}."
                         )
                         operation = "airdrop_gift"
+                    elif (
+                        inout == "incoming"
+                        and asset == "LUNC"
+                        and asset_class == "Fiat"
+                        and utc_time.year == 2022
+                        and utc_time.month == 5
+                    ):
+                        # In May 2022 the Terra (LUNA) blockchain crashed. In response, a new chain
+                        # Terra 2.0 (LUNA) was created. The new old chain is still tradeable as
+                        # Terra Classic (LUNC) and holders of LUNA before the crash received their
+                        # LUNC tokens as airdrop. This also applied to LUNA tokens held through
+                        # bitpanda crypto indices.
+                        # Source for bitpanda LUNC airdrop:
+                        # https://support.bitpanda.com/hc/en-us/articles/4995318011292-Terra-2-0-LUNA-Airdrop
+                        #
+                        # The German law regarding this case is not entirely clear:
+                        # https://www.winheller.com/bankrecht-finanzrecht/bitcointrading/bitcoinundsteuer/besteuerung-hardforks-ledger-splits.html
+                        # TODO: This should actually copy the history from the original LUNA history.
+                        log.warning(
+                            f"WARNING: Airdrop of {asset} is a result of the fork of the "
+                            f"LUNA blockchain in May 2022. The legal status of "
+                            f"taxation of hardfork results is not clear in German law. "
+                            f"Also, the date of procurement should be set to the date(s) "
+                            f"of procurement of the original coins, essentially copying "
+                            f"the history of the original chain, which is NOT YET IMPLEMENTED. "
+                            f"See https://support.bitpanda.com/hc/en-us/articles/4995318011292-Terra-2-0-LUNA-Airdrop "
+                            f"for more information. "
+                            f"Please open an issue or PR if you know how to resolve this. "
+                            f"In row {row} in file {file_path}."
+                        )
+                        # Rewrite this asset_class because "Fiat" clearly wrong.
+                        asset_class = "Cryptocurrency"
+                        operation = "airdrop_gift"
+                    elif (
+                        inout == "incoming"
+                        and asset_class == "Cryptocurrency"
+                        and asset != "BEST"
+                        and utc_time < datetime.datetime(2022, 6, 14, 0, 0, 0, 0, utc_time.tzinfo)
+                    ):
+                        # Bitpanda tagged incoming staking rewards as incoming transfer until June 14 2022
+                        # or a few days before that date. After that, staking rewards are correctly tagged as "reward".
+                        operation = "reward"
                     else:
                         log.warning(
                             f"'Transfer' operations are not "

src/price_data.py

@@ -355,8 +355,10 @@ def _get_price_bitpanda_pro(
                     )
                 r = requests.get(baseurl, params=params)
 
-                assert r.status_code == 200, f"No valid response from ONE TRADING (ex Bitpanda Pro) API\nError: {r.json()['error']}"
                 data = r.json()
+                if r.status_code == 400 and data["error"] == f"The requested market {base_asset}_{quote_asset} is not available.":
+                    raise ValueError(data["error"])
+                assert r.status_code == 200, f"No valid response from ONE TRADING (ex Bitpanda Pro) API\nError: {r.json()['error']}"
 
                 # exit loop if data is valid
                 if data:
@@ -380,11 +382,12 @@ def _get_price_bitpanda_pro(
             raise RuntimeError
 
         # this should never be triggered, but just in case assert received data
-        assert data, f"No valid price data for {base_asset} / {quote_asset} at {end}"
+        assert data["candlesticks"], f"No valid price data for {base_asset} / {quote_asset} at {end}"
+        data = data["candlesticks"]
 
-        # simply take the average of the latest data element
-        high = misc.force_decimal(data[-1]["high"])
-        low = misc.force_decimal(data[-1]["low"])
+        # simply take the average of the first data element
+        high = misc.force_decimal(data[0]["high"])
+        low = misc.force_decimal(data[0]["low"])
 
         # if spread is greater than 3%
         if (high - low) / high > 0.03:
@@ -607,24 +610,39 @@ def get_cost(
         reference_coin: str = config.FIAT,
     ) -> decimal.Decimal:
         op = op_sc if isinstance(op_sc, tr.Operation) else op_sc.op
-        if op.coin in ["ETHW", "BEST"] and op.platform == "bitpanda":
-            # ETHW and BEST are not available via ONE TRADING (ex Bitpanda Pro) API
-            # => use the price from the exported data.
-            if op.exported_price is not None:
-                return op.exported_price
-            if op.coin == "BEST" and isinstance(op, tr.Fee):
+        try:
+            price = self.get_price(op.platform, op.coin, op.utc_time, reference_coin)
+        except ValueError as e:
+            log.warning(
+                f"The API didn't provide a valid response. Using the price from the csv file if possible.\n"
+                f"\t\tCoin: {op.coin} | Op: {type(op).__name__} | Platform: {op.platform} | Row: {op.line} | File: {op.file_path}\n"
+                f"\t\tCaught exception: {e}"
+            )
+            if op.platform == "bitpanda":
+                # LUNC, ETHW, BEST and maybe more are not available via ONE TRADING (ex Bitpanda Pro) API
+                # => use the price from the exported data.
+                if op.exported_price is not None:
+                    price = op.exported_price
+
                 # Fees paid with BEST don't have a value given in the exported data.
                 # The value also can't be queried from the ONE TRADING (ex Bitpanda Pro) API (anymore)
+                if op.coin == "BEST" and isinstance(op, tr.Fee):
+                    log.warning(
+                        f"Can't get price for '{type(op).__name__}' of {op.coin} on platform {op.platform} anymore.\n"
+                        f"A withdrawal of BEST on bitpanda is likely a deduction of fees. For now we'll assume a value of 0.\n"
+                        f"For accurately calculating fees, this needs to be fixed. PRs welcome!\n"
+                        f"(row {op.line} in {op.file_path}"
+                    )
+                    return 0
+            else:
                 log.warning(
-                    f"Can't get price for '{type(op).__name__}' of {op.coin} on platform {op.platform} for operation 'Withdrawal' anymore.\n"
-                    f"A withdrawal of BEST on bitpanda is likely a deduction of fees. For now we'll assume a value of 0.\n"
-                    f"For accurately calculating fees, this needs to be fixed. PRs welcome!\n"
-                    f"(row {op.line} in {op.file_path}"
+                    f"Could not get any price info for {type(op).__name__} {op.coin} on {op.platform}! "
+                    f"Row: {op.line} | File: {op.file_path}"
                 )
-                return 0
-            raise RuntimeError(f"Can't get price for '{type(op).__name__}' of {op.coin} on platform {op.platform} (row {op.line} in {op.file_path})!")
-        else:
-            price = self.get_price(op.platform, op.coin, op.utc_time, reference_coin)
+                raise RuntimeError(e)
+
+        # This may fail if an exchange is queried for a non existant coin/fiat pair and the operation doesn't include an exported price.
+        assert price, f"Could not get a price for asset {op.coin} at {op.utc_time}"
 
         if isinstance(op_sc, tr.Operation):
             return price * op_sc.change

src/taxman.py

@@ -261,8 +261,8 @@ def _evaluate_sell(
         Raises:
             NotImplementedError: When there are more than two different fee coins.
         """
-        assert op.coin == sc.op.coin
-        assert op.change >= sc.sold
+        assert op.coin == sc.op.coin, f"Error evaluating op.coin==sc.op.coin:\n\t\t{op}\n\t\t{sc}"
+        assert op.change >= sc.sold, f"Error evaluating op.change >=sc.sold:\n\t\t{op}\n\t\t{sc}"
 
         # Share the fees and sell_value proportionally to the coins sold.
         percent = sc.sold / op.change
@@ -284,7 +284,7 @@ def _evaluate_sell(
         except Exception as e:
             if ReportType is tr.UnrealizedSellReportEntry:
                 log.warning(
-                    "Catched the following exception while trying to query an "
+                    "Caught the following exception while trying to query an "
                     f"unrealized sell value for {sc.sold} {sc.op.coin} at deadline "
                     f"on platform {sc.op.platform}. "
                     "If you want to see your unrealized sell value "
@@ -294,7 +294,7 @@ def _evaluate_sell(
                     "The sell value for this calculation will be set to 0. "
                     "Your unrealized sell summary will be wrong and will not "
                     "be exported.\n"
-                    f"Catched exception: {e}"
+                    f"Caught exception: {type(e).__name__}: {e}"
                 )
                 sell_value_in_fiat = decimal.Decimal()
                 self.unrealized_sells_faulty = True