-
Notifications
You must be signed in to change notification settings - Fork 17
feat: Integrate Bitget for Rate Fetching & Median Calculation Across Multiple Providers #441
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sundayonah
wants to merge
57
commits into
main
Choose a base branch
from
biget-rates-market-refactor
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 13 commits
Commits
Show all changes
57 commits
Select commit
Hold shift + click to select a range
f0185fa
feat: implement multi-provider rate fetching system with test
sundayonah 469f49f
feat(rates): enhance rate fetching with detailed metadata
sundayonah 9d00340
test: modify test and move external_market file to utils
sundayonah 7f72384
refactor(config): move API URLs to environment variables
sundayonah bd3c348
refactor(utils): overhaul external market rate fetching implementation
sundayonah 31e3a32
refactor: (utils) replace FetchBitgetRate endpoint, remove /api/v1 fr…
sundayonah 59e047c
fix: resolve rebase issues and reset branch to stable state
sundayonah fe0cdce
refactor(config): remove Bitget API keys from AuthConfiguration
sundayonah 4864dea
Merge branch 'main' into biget-rates-market-refactor
sundayonah 65c7c58
fix: update function name for fetching Quidax rates
sundayonah 0a46c74
Merge branch 'main' into biget-rates-market-refactor
sundayonah b9116be
Merge branch 'main' into biget-rates-market-refactor
sundayonah 984efba
Merge branch 'main' into biget-rates-market-refactor
chibie 88f601e
feat: add HTTPS URL validation for host identifier in UpdateProviderP…
sundayonah 4cba6d3
test: add vtest cases for HostIdentifier HTTPS validation
sundayonah 3412c04
fix(tests): update HostIdentifier to use HTTPS and improve assertions…
sundayonah a0108f6
fix: setup repo
Atanda1 7959afd
docs: update README to specify Sender API usage with sandbox API Key …
chibie fda12a3
fix: update SQL dump and user operation logic
chibie ae9148b
fix: update SQL dump for provider profile host identifier
chibie 3fabc7f
chore: remove unnecessary print statements
chibie cd9feb2
fix: Handle amounts below minimum bucket
sundayonah bf6c876
test: Add unit tests for minimum bucket handling
sundayonah a0896a6
fix: remove duplicate token import
sundayonah dd14d9b
feat: modify provision bucket call to capture unused isLessThanMin an…
sundayonah d0ba090
fix(indexer): improve error handling in CreateLockPaymentOrder and en…
chibie b87ad09
fix(indexer): enhance error message in CreateLockPaymentOrder to incl…
chibie f0e4e88
fix(indexer): change error handling for token fetch in CreateLockPaym…
chibie 61beac8
Provider Token Rate Slippage Configuration (#415)
sundayonah 7161a65
feat: enhance logging with contextual information across services (#459)
onahprosper 4823053
refactor(logging): standardize error logging format and enhance conte…
chibie 6893643
refactor(profile): update provider rate calculation logic in profile …
chibie c966593
refactor(logging): standardize error logging format across indexer an…
chibie c5bdb59
refactor(indexer, priority_queue, order): enforce settlement address …
chibie 2007b52
fix(profile): handle rate slippage assignment in provider profile upd…
chibie f4a9137
fix(priority_queue): change error handling in matchRate to continue o…
chibie eed9fb0
refactor(provider): improve stats calculation and error handling in P…
chibie 4f8bcb3
refactor(provider): streamline stats calculation for USD and local st…
chibie 8a2baff
refactor(provider): update stats query to filter by USD token for acc…
chibie aa985f7
refactor(provider): simplify total fiat volume calculation in Stats m…
chibie e0aa4c8
refactor(sender, indexer): enhance institution retrieval logic and im…
chibie eae210f
refactor(provider): enhance Stats method to filter out USD token and …
chibie eca9a66
refactor(sender): remove debug print statements from Stats method for…
chibie 861a467
feat(migrations): add TZS and UGX bank institutions
sundayonah e5702f9
fix: remove provision buckets from UGX and TZS bank institution migra…
sundayonah 21525d6
refactor(profile): update profile test and API response messages for …
chibie 0a8b789
refactor(indexer): commented out the split lock payment order functio…
chibie e198963
chore(migrations): updated versioning of TZS and UGX bank institution…
chibie 94ee460
fix: timezone error in Slack notifications with additional test cases
sundayonah 4a7dd78
feat: support ~all countries in KYC verification (#457)
sundayonah 5659d9c
refactor: update KYC verification structure and remove deprecated files
chibie 0166b0a
feat: add metadata field to payment order and recipient models (#467)
chibie d3f80fd
feat(ci): add Atlas database migrations workflow (#468)
chibie 674b9e2
refactor: enhance error logging in FulfillOrder and CancelOrder contr…
onahprosper 99575f2
refactor(utils): overhaul external market rate fetching implementation
sundayonah ba44653
fix: resolve rebase issues and reset branch to stable state
sundayonah 5257afa
fix: remove duplicate type definitions
sundayonah File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,217 @@ | ||
package utils | ||
|
||
import ( | ||
"bytes" | ||
"encoding/json" | ||
"fmt" | ||
"io" | ||
"net/http" | ||
"strings" | ||
"time" | ||
|
||
fastshot "github.com/opus-domini/fast-shot" | ||
"github.com/paycrest/aggregator/types" | ||
"github.com/shopspring/decimal" | ||
) | ||
|
||
var ( | ||
BitgetAPIURL = "https://www.bitget.com" | ||
BinanceAPIURL = "https://p2p.binance.com" | ||
QuidaxAPIURL = "https://www.quidax.com" | ||
) | ||
|
||
// FetchExternalRate fetches the external rate for a fiat currency | ||
func FetchExternalRate(currency string) (decimal.Decimal, error) { | ||
currency = strings.ToUpper(currency) | ||
supportedCurrencies := []string{"KES", "NGN", "GHS", "TZS", "UGX", "XOF"} | ||
isSupported := false | ||
for _, supported := range supportedCurrencies { | ||
if currency == supported { | ||
isSupported = true | ||
break | ||
} | ||
} | ||
if !isSupported { | ||
return decimal.Zero, fmt.Errorf("ComputeMarketRate: currency not supported") | ||
} | ||
|
||
var prices []decimal.Decimal | ||
|
||
// Fetch rates based on currency | ||
if currency == "NGN" { | ||
quidaxRate, err := FetchQuidaxRates(currency) | ||
if err == nil { | ||
prices = append(prices, quidaxRate) | ||
} | ||
} else { | ||
binanceRates, err := FetchBinanceRates(currency) | ||
if err == nil { | ||
prices = append(prices, binanceRates...) | ||
} | ||
} | ||
|
||
// Fetch Bitget rates for all supported currencies | ||
bitgetRates, err := FetchBitgetRates(currency) | ||
if err == nil { | ||
prices = append(prices, bitgetRates...) | ||
} | ||
|
||
if len(prices) == 0 { | ||
return decimal.Zero, fmt.Errorf("ComputeMarketRate: no valid rates found") | ||
} | ||
|
||
// Return the median price | ||
return Median(prices), nil | ||
} | ||
|
||
// FetchQuidaxRate fetches the USDT exchange rate from Quidax (NGN only) | ||
func FetchQuidaxRates(currency string) (decimal.Decimal, error) { | ||
url := fmt.Sprintf("/api/v1/markets/tickers/usdt%s", strings.ToLower(currency)) | ||
|
||
res, err := fastshot.NewClient(QuidaxAPIURL). | ||
Config().SetTimeout(30*time.Second). | ||
Build().GET(url). | ||
Retry().Set(3, 5*time.Second). | ||
Send() | ||
if err != nil { | ||
return decimal.Zero, fmt.Errorf("FetchQuidaxRate: %w", err) | ||
} | ||
|
||
data, err := ParseJSONResponse(res.RawResponse) | ||
if err != nil { | ||
return decimal.Zero, fmt.Errorf("FetchQuidaxRate: %w", err) | ||
} | ||
|
||
price, err := decimal.NewFromString(data["data"].(map[string]interface{})["ticker"].(map[string]interface{})["buy"].(string)) | ||
if err != nil { | ||
return decimal.Zero, fmt.Errorf("FetchQuidaxRate: %w", err) | ||
} | ||
|
||
return price, nil | ||
} | ||
|
||
// FetchBinanceRates fetches USDT exchange rates from Binance P2P | ||
func FetchBinanceRates(currency string) ([]decimal.Decimal, error) { | ||
res, err := fastshot.NewClient(BinanceAPIURL). | ||
Config().SetTimeout(30*time.Second). | ||
Header().Add("Content-Type", "application/json"). | ||
Build().POST("/bapi/c2c/v2/friendly/c2c/adv/search"). | ||
Retry().Set(3, 5*time.Second). | ||
Body().AsJSON(map[string]interface{}{ | ||
"asset": "USDT", | ||
"fiat": currency, | ||
"tradeType": "SELL", | ||
"page": 1, | ||
"rows": 20, | ||
}). | ||
Send() | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBinanceRates: %w", err) | ||
} | ||
|
||
resData, err := ParseJSONResponse(res.RawResponse) | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBinanceRates: %w", err) | ||
} | ||
|
||
data, ok := resData["data"].([]interface{}) | ||
if !ok || len(data) == 0 { | ||
return nil, fmt.Errorf("FetchBinanceRates: no data in response") | ||
} | ||
|
||
var prices []decimal.Decimal | ||
for _, item := range data { | ||
adv, ok := item.(map[string]interface{})["adv"].(map[string]interface{}) | ||
if !ok { | ||
continue | ||
} | ||
|
||
price, err := decimal.NewFromString(adv["price"].(string)) | ||
if err != nil { | ||
continue | ||
} | ||
|
||
prices = append(prices, price) | ||
} | ||
|
||
if len(prices) == 0 { | ||
return nil, fmt.Errorf("FetchBinanceRates: no valid prices found") | ||
} | ||
|
||
return prices, nil | ||
} | ||
|
||
// FetchBitgetRates fetches USDT exchange rates from Bitget P2P listings | ||
func FetchBitgetRates(currency string) ([]decimal.Decimal, error) { | ||
payload := map[string]interface{}{ | ||
"side": 2, | ||
"pageNo": 1, | ||
"pageSize": 20, | ||
"coinCode": "USDT", | ||
"fiatCode": currency, | ||
"languageType": 0, | ||
} | ||
|
||
payloadBytes, err := json.Marshal(payload) | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBitgetRates: failed to marshal payload: %w", err) | ||
} | ||
|
||
client := &http.Client{Timeout: 30 * time.Second} | ||
req, err := http.NewRequest("POST", BitgetAPIURL+"/v1/p2p/pub/adv/queryAdvList", bytes.NewBuffer(payloadBytes)) | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBitgetRates: failed to create request: %w", err) | ||
} | ||
req.Header.Set("Content-Type", "application/json") | ||
|
||
var resp *http.Response | ||
err = Retry(3, 5*time.Second, func() error { | ||
var retryErr error | ||
resp, retryErr = client.Do(req) | ||
return retryErr | ||
}) | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBitgetRates: failed to send request after retries: %w", err) | ||
} | ||
defer resp.Body.Close() | ||
|
||
bodyBytes, err := io.ReadAll(resp.Body) | ||
if err != nil { | ||
return nil, fmt.Errorf("FetchBitgetRates: failed to read response body: %w", err) | ||
} | ||
|
||
var resData types.BitgetResponse | ||
err = json.Unmarshal(bodyBytes, &resData) | ||
if err != nil { | ||
fmt.Printf("FetchBitgetRates: failed to parse response, raw body: %s\n", string(bodyBytes)) | ||
return nil, fmt.Errorf("FetchBitgetRates: failed to parse response: %w", err) | ||
} | ||
|
||
if resData.Code != "00000" { | ||
return nil, fmt.Errorf("FetchBitgetRates: API error: %s", resData.Msg) | ||
} | ||
|
||
if len(resData.Data.DataList) == 0 { | ||
return nil, fmt.Errorf("FetchBitgetRates: no sell ads found for %s/USDT", currency) | ||
} | ||
|
||
var prices []decimal.Decimal | ||
for i, ad := range resData.Data.DataList { | ||
if ad.CoinCode != "USDT" || ad.FiatCode != currency { | ||
continue | ||
} | ||
price, err := decimal.NewFromString(ad.Price) | ||
if err != nil { | ||
fmt.Printf("FetchBitgetRates: skipping ad at index %d with invalid price '%s': %v\n", i, ad.Price, err) | ||
continue | ||
} | ||
prices = append(prices, price) | ||
} | ||
|
||
if len(prices) == 0 { | ||
return nil, fmt.Errorf("FetchBitgetRates: no valid sell ads found for %s/USDT", currency) | ||
} | ||
|
||
return prices, nil | ||
} | ||
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check is not enough.
The first check should includes what we have here
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
based on this part:
GIVEN the currency is NGN
WHEN fetching rates for USDT/NGN
THEN the system should fetch rates from Quidax and Bitget
GIVEN the currency is not NGN
WHEN fetching rates for USDT/
THEN the system should fetch rates from Binance and Bitget
if currency == NGN use
FetchQuidaxRates
andFetchBitgetRates
else if currency is ... use
FetchBinanceRates
andFetchBitgetRates