Compare commits

...

100 Commits

Author SHA1 Message Date
MaysWind 4d9643dcb2 raise an error if the file does not exist when uploading an artifact 2025-09-27 17:20:45 +08:00
MaysWind 5dc4ad60ba remove unnecessary parameter 2025-09-27 17:12:41 +08:00
MaysWind 2e5dd7d513 add content encoding to the response headers when the server returns js, csv or tsv 2025-09-27 17:00:42 +08:00
MaysWind efe088f591 update README.md 2025-09-27 16:19:27 +08:00
MaysWind d5016e853e update badges 2025-09-27 16:00:47 +08:00
MaysWind d334bd7b9a automatically create release via github action 2025-09-27 16:00:41 +08:00
MaysWind 21edf0157a modify file name 2025-09-27 15:59:45 +08:00
MaysWind 388167705a build package for windows via github actions 2025-09-27 15:59:14 +08:00
MaysWind 2423b37cbb update uploading artifact job 2025-09-27 14:46:48 +08:00
MaysWind 786796d457 update uploading artifact job 2025-09-27 08:14:36 +08:00
MaysWind 0ed9216260 create tar.gz packages via github actions 2025-09-26 23:20:09 +08:00
MaysWind eb13f10121 move enable_two_factor, enable_forget_password and forget_password_require_email_verify option to the auth section 2025-09-26 21:51:26 +08:00
MaysWind 76ce6f6f9c fix incorrect comment 2025-09-26 21:34:34 +08:00
MaysWind eb305139f5 fix the locale settings on the profile page does not immediately update the preview 2025-09-25 23:32:02 +08:00
MaysWind 8df73f202a update splash screen image file name 2025-09-24 23:11:50 +08:00
MaysWind c22751de6f add new translation contributor 2025-09-24 23:11:41 +08:00
MaysWind c3f1cb0c61 update transaction 2025-09-24 23:11:27 +08:00
MaysWind fc1fc58aa1 update locale default settings 2025-09-24 22:40:38 +08:00
Natthavat Sukdumrongpreecha e4b5e96534 fix: update var name from en to th 2025-09-24 12:25:31 +08:00
Natthavat Sukdumrongpreecha 66303a8965 feat: translate to thai language 2025-09-24 12:25:31 +08:00
MaysWind 9589dd2486 add github issue template 2025-09-23 01:39:18 +08:00
MaysWind 3d5b887e23 support Google AI LLM provider 2025-09-23 00:50:44 +08:00
MaysWind b967a214cb code refactor 2025-09-23 00:28:17 +08:00
MaysWind 5a9877588f improve date time recognition in AI image recognition 2025-09-22 22:21:35 +08:00
MaysWind fc5f8e4633 update go.mod 2025-09-22 22:04:34 +08:00
MaysWind 028bca50ea renamed structs and interfaces to reduce ambiguity 2025-09-22 22:04:06 +08:00
MaysWind 6853bbfb68 renamed structs and interfaces to reduce ambiguity 2025-09-22 22:02:37 +08:00
MaysWind d4fee27a3d fix the result of detecting whether a transaction has been modified is wrong when no default account is set 2025-09-22 00:59:10 +08:00
MaysWind 245fdd78e4 clear all transactions of specified account (#228) 2025-09-22 00:26:59 +08:00
MaysWind cbe784172e fix cannot connect PostgreSQL via Unix socket (#253) 2025-09-21 22:00:22 +08:00
MaysWind bf21e45cba update description of query_transactions mcp tool json scheme 2025-09-21 21:00:14 +08:00
MaysWind 359c430a39 add new translation contributor 2025-09-21 20:53:19 +08:00
MaysWind 669a217180 improve the style of long tips and drag-and-drop image 2025-09-21 20:48:50 +08:00
MaysWind e9507241ed update transaction 2025-09-21 20:34:17 +08:00
brieucdlf f2536749f6 feat(locale): add French translation 2025-09-21 19:05:52 +08:00
MaysWind 118558d25b modify the structure of the large language model options in the configuration file 2025-09-21 17:49:49 +08:00
MaysWind d9cd270ff4 don't save draft when creating transactions through AI image recognition in mobile version 2025-09-21 15:51:41 +08:00
MaysWind 9dee449f10 fix the incorrect mouse pointer style during image recognition 2025-09-21 15:31:53 +08:00
MaysWind ae19ca4383 improved the experience when no transaction information is detected 2025-09-21 15:28:26 +08:00
MaysWind 32fed8d6fb fix the application could not start if no LLM provider was configured 2025-09-21 15:16:34 +08:00
MaysWind ec325c9e6b fix AI image recognition limit could not be set for users 2025-09-21 15:15:50 +08:00
MaysWind 5fbb29abd3 modify style 2025-09-21 12:29:04 +08:00
MaysWind f06c6523a2 add capture=camera to the accept attribute in file input when selecting transaction pictures 2025-09-21 12:00:19 +08:00
MaysWind 02514fc457 use system control to take photo or select an image 2025-09-21 12:00:11 +08:00
MaysWind 5d88287ae2 create transactions from AI receipt image recognition results 2025-09-21 04:00:57 +08:00
MaysWind 00f1d0418f show execution error in preview area 2025-09-20 22:54:10 +08:00
MaysWind 18b270debb support for disabling the app’s built-in swipe-back navigation (#249) 2025-09-20 22:02:18 +08:00
MaysWind d947164eb6 add unit test 2025-09-20 21:34:13 +08:00
MaysWind 1a1bb6077c hidden transaction tags are not allowed when importing transactions or using the add_transaction mcp tool 2025-09-20 21:22:44 +08:00
MaysWind 05b5cab12b fix the add_transaction mcp tool set wrong categories from the wrong category type 2025-09-20 21:19:41 +08:00
MaysWind a82fdd4946 add unit tests 2025-09-20 21:08:42 +08:00
MaysWind 4def7ed60c the query_transactions_tool_handler mcp tool supports filtering multiple categories or accounts with the same name, and filtering sub-accounts / secondary categories by their parent account / category name 2025-09-20 21:08:34 +08:00
MaysWind d50ce0140f code refactor 2025-09-20 02:08:39 +08:00
MaysWind 51678aee04 don't return hidden tags for query_all_transaction_tags mcp tool 2025-09-20 00:42:42 +08:00
MaysWind 019689087d code refactor 2025-09-19 23:40:37 +08:00
MaysWind 0c1d77f7ae modify text 2025-09-19 21:35:48 +08:00
MaysWind 8de51e6e71 show the selected count even when the number of items is less than 10 2025-09-19 00:27:07 +08:00
MaysWind dc993da218 import investment transactions from alipay statement file (#246) 2025-09-19 00:18:19 +08:00
MaysWind 983f7fec0f fix can not replace empty transaction categories when using batch rules 2025-09-18 22:51:00 +08:00
MaysWind ce74c4817b fix transaction type was not checked when replacing transaction categories using batch rules (#248) 2025-09-18 22:42:30 +08:00
MaysWind bc363438f1 move the request_id_header option to the server section and the enable_two_factor option to the user section 2025-09-17 21:16:49 +08:00
MaysWind 979b16d520 display time numbers with fixed width style in the mobile version time picker 2025-09-17 21:07:22 +08:00
MaysWind 9686eb020f fix cannot import OFX file which has only one but very long line (#243) 2025-09-15 22:51:16 +08:00
MaysWind 88dea9acaa use the current browser's time zone as the sample 2025-09-15 00:38:39 +08:00
MaysWind c75fdfea1c support custom script to process delimiter-separated values (data) file / data 2025-09-15 00:21:14 +08:00
MaysWind 538d2b8205 the add transaction page of mobile version supports the destinationAccountId and comment parameters 2025-09-14 17:41:21 +08:00
MaysWind 30d36a3b07 add noUncheckedIndexedAccess compiler options 2025-09-14 17:29:42 +08:00
MaysWind 95bcd8e4c8 upgrade third party dependencies 2025-09-14 17:29:23 +08:00
MaysWind 1a8ce7d58d use for-of statements to replace for and for-in 2025-09-14 17:18:47 +08:00
MaysWind 4700446ca0 use for-of statements to replace for and for-in 2025-09-14 01:43:04 +08:00
MaysWind 67bc81d3e2 fix user custom exchange rates update page /dialog could not be opened 2025-09-14 01:17:46 +08:00
MaysWind 878a3a018e modify style 2025-09-14 01:11:02 +08:00
MaysWind e463c2dc95 batch adding transaction tags in import transaction tool 2025-09-13 23:05:26 +08:00
MaysWind 422cf49517 batch removing specified tag or batch removing invalid tag in import transaction tool 2025-09-13 23:00:39 +08:00
MaysWind 77d2426c14 code refactor 2025-09-13 21:17:55 +08:00
MaysWind 1c4dc55bb6 modify style 2025-09-13 15:37:25 +08:00
MaysWind ba72f421dc code refactor 2025-09-13 15:34:20 +08:00
MaysWind 36d1e01008 fix the display format of the fiscal year start date not updated after changing the number system on user profile page 2025-09-13 02:19:57 +08:00
MaysWind e52c7037c7 import transactions from JD.com finance statement file (#240) 2025-09-13 01:53:41 +08:00
MaysWind f5235ba08e modify text 2025-09-13 01:53:41 +08:00
xiaolvdou adc4899ea6 fix MCP protocol version requirement 2025-09-11 10:40:34 +08:00
MaysWind 34c5a1750e use for-of statements to replace for and for-in 2025-09-09 23:48:42 +08:00
MaysWind c75a902d84 fix could not select category name via checkbox in the batch create category dialog 2025-09-09 23:22:58 +08:00
MaysWind 7e2e1a4ad3 use truncation instead of rounding down or rounding to the nearest value when numerical calculations exceed precision limits 2025-09-09 20:46:51 +08:00
MaysWind d4603a1892 code refactor 2025-09-09 00:01:15 +08:00
MaysWind 642e51bc0c use and display the Gregorian calendar when calculating months, quarters, years, and fiscal years 2025-09-08 00:32:30 +08:00
MaysWind 5591abdb3b upgrade third party dependencies 2025-09-07 23:14:59 +08:00
MaysWind ce9378c43f support filtering accounts and transaction categories for overview in home page (#209) 2025-09-07 13:57:07 +08:00
MaysWind 3ae72623ad code refactor 2025-09-07 00:15:35 +08:00
MaysWind affc02655b remove redundant spaces 2025-09-06 23:34:35 +08:00
MaysWind a469d66358 calendar display type supports Gregorian with Persian, date display type supports Persian calendar 2025-09-06 23:34:29 +08:00
MaysWind 757f9e5b02 fix the toast text size and the text input size in dialogs on the mobile version did not adjust according to the font size settings 2025-09-06 01:18:05 +08:00
MaysWind 8368b02be8 calendar display type supports Gregorian with Chinese 2025-09-06 01:06:31 +08:00
MaysWind e15a5617e6 support dates with YYYYMMDD format when importing delimiter-separated values file / data (#219) 2025-09-04 22:04:14 +08:00
MaysWind f604b2c766 upgrade golang to 1.25.1, node.js to 24.7.0 2025-09-04 21:57:19 +08:00
MaysWind d6dc9f8170 fix the checkbox in the scheduled transaction frequency dropdown list couldn’t be used to select options (#225) 2025-09-03 19:50:00 +08:00
MaysWind a71be1bf05 fix cannot delete accounts / transaction categories and tags when using postgres db (#218) 2025-09-01 22:44:34 +08:00
MaysWind bcf11631d6 use integers to calculate formulas for beancount amount formula 2025-09-01 01:16:58 +08:00
MaysWind 989183c8be use integers to calculate formulas for evaluator (#214) 2025-09-01 00:32:14 +08:00
MaysWind 8bd0fd88af bump version to 1.1.0 2025-08-31 13:25:47 +08:00
280 changed files with 57943 additions and 6049 deletions
+70
View File
@@ -0,0 +1,70 @@
name: Bug Report
description: Report a bug in ezBookkeeping
labels: bug
body:
- type: checkboxes
id: checkboxes
attributes:
label: Before You Submit
description: Please check whether the following items have been completed.
options:
- label: I've already checked this bug hasn't been raised in [issues](https://github.com/mayswind/ezbookkeeping/issues)
required: true
- type: textarea
id: description
attributes:
label: Description
description: Please provide a brief description of this bug.
validations:
required: true
- type: textarea
id: reproduction-steps
attributes:
label: Steps to reproduce
description: Please describe the steps to reproduce this bug.
placeholder: |
1.
2.
3.
validations:
required: true
- type: input
id: ezbookkeeping-version
attributes:
label: ezBookkeeping Version
description: ezBookkeeping version and commit hash of your instance, e.g. "v1.0.0 (20e2444)"
validations:
required: true
- type: input
id: server-os
attributes:
label: Server Operating System
description: The operating system information you are using to deploy ezBookkeeping, e.g "Debian GNU/Linux 11 amd64"
- type: input
id: server-database
attributes:
label: Database
description: The database system you are using, e.g. "MariaDB v11.7.2"
- type: dropdown
id: reproduce-on-demo-site
attributes:
label: Can you reproduce this bug on the ezBookkeeping demo site?
description: |
ezBookkeeping demo site: https://ezbookkeeping-demo.mayswind.net/
options:
- "No"
- "Yes"
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional information
description: If you can, provide any related screenshots or logs here.
+1
View File
@@ -0,0 +1 @@
blank_issues_enabled: false
@@ -0,0 +1,26 @@
name: Feature Request
description: Request a feature or enhancement for ezBookkeeping
labels: enhancement
body:
- type: checkboxes
id: checkboxes
attributes:
label: Before You Submit
description: Please check whether the following items have been completed.
options:
- label: I've already checked this request hasn't been raised in [issues](https://github.com/mayswind/ezbookkeeping/issues)
required: true
- type: textarea
id: description
attributes:
label: Feature Description
description: Please describe your feature request.
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional information
description: If you can, provide any other context or screenshots about this feature request here.
@@ -1,4 +1,4 @@
name: Docker Snapshot
name: Build for Non-Main Branches
on:
push:
@@ -6,7 +6,7 @@ on:
- main
jobs:
build:
build-linux-docker:
runs-on: ubuntu-latest
steps:
-
+224
View File
@@ -0,0 +1,224 @@
name: Build Release
on:
push:
tags:
- "v*.*.*"
jobs:
build-linux-docker:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.meta.outputs.version }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ secrets.DOCKER_USERNAME }}/ezbookkeeping
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v8.1.5
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v6
with:
file: Dockerfile
context: .
platforms: |
linux/amd64
linux/arm64/v8
linux/arm/v7
linux/arm/v6
push: true
build-args: |
RELEASE_BUILD=1
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
upload-linux-artifact:
needs: build-linux-docker
runs-on: ubuntu-latest
strategy:
matrix:
include:
- arch: linux/amd64
arch_alias: linux-amd64
- arch: linux/arm64/v8
arch_alias: linux-arm64
- arch: linux/arm/v7
arch_alias: linux-armv7
- arch: linux/arm/v6
arch_alias: linux-armv6
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Pull and save packaged files for ${{ matrix.arch }}
run: |
VERSION=${{ needs.build-linux-docker.outputs.image-tag }}
IMAGE=${{ secrets.DOCKER_USERNAME }}/ezbookkeeping:${VERSION}
docker pull --platform ${{ matrix.arch }} ${IMAGE}
cid=$(docker create "${IMAGE}")
docker cp ${cid}:/ezbookkeeping ./ezbookkeeping
docker rm ${cid}
cd ezbookkeeping
tar -czf ../ezbookkeeping-v${VERSION}-${{ matrix.arch_alias }}.tar.gz *
cd ..
rm -rf ezbookkeeping
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-${{ matrix.arch_alias }}
path: ezbookkeeping-${{ github.ref_name }}-${{ matrix.arch_alias }}.tar.gz
if-no-files-found: error
build-and-upload-windows-package:
needs: upload-linux-artifact
runs-on: windows-latest
env:
GO_VERSION: "1.25.1"
MINGW_VERSION: "14.2.0"
MINGW_REVISION: "v12-rev2"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Download linux-amd64 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-linux-amd64
path: artifacts
- name: Extract frontend files from linux-amd64 package
run: |
New-Item -ItemType Directory -Path package
tar -xzf (Get-ChildItem artifacts\ezbookkeeping-${{ github.ref_name }}-linux-amd64.tar.gz) -C package
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
- name: Install MinGW
run: |
$mingwVersion = "${{ env.MINGW_VERSION }}"
$mingwRevision = "${{ env.MINGW_REVISION }}"
$url = "https://github.com/niXman/mingw-builds-binaries/releases/download/${mingwVersion}-rt_${mingwRevision}/x86_64-${mingwVersion}-release-posix-seh-ucrt-rt_${mingwRevision}.7z"
$archive = "C:\mingw.7z"
$mingwDir = "C:\mingw64"
Write-Host "Downloading MinGW from ${url}"
Invoke-WebRequest -Uri ${url} -OutFile ${archive}
Remove-Item -Recurse -Force ${mingwDir}
New-Item -ItemType Directory -Path ${mingwDir}
Write-Host "Extracting MinGW to ${mingwDir}"
7z x ${archive} -oC:\
"${mingwDir}\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Build backend for windows-x64
env:
RELEASE_BUILD: "1"
BUILD_PIPELINE: "1"
CHECK_3RD_API: ${{ vars.CHECK_3RD_API }}
SKIP_TESTS: ${{ vars.SKIP_TESTS }}
run: |
.\build.ps1 backend
- name: Package Windows build
run: |
New-Item -ItemType Directory -Path "ezbookkeeping"
New-Item -ItemType Directory -Path "ezbookkeeping\data"
New-Item -ItemType Directory -Path "ezbookkeeping\storage"
New-Item -ItemType Directory -Path "ezbookkeeping\log"
Copy-Item ezbookkeeping.exe -Destination ezbookkeeping\
Copy-Item -Recurse -Force package\public -Destination ezbookkeeping\public
Copy-Item -Recurse -Force conf -Destination ezbookkeeping\conf
Copy-Item -Recurse -Force templates -Destination ezbookkeeping\templates
Copy-Item .\LICENSE -Destination ezbookkeeping\
Push-Location ezbookkeeping
7z a -r -tzip -mx9 ..\ezbookkeeping-${{ github.ref_name }}-windows-x64.zip *
Pop-Location
Remove-Item -Recurse -Force ezbookkeeping
- name: Upload Windows artifact
uses: actions/upload-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-windows-x64
path: ezbookkeeping-${{ github.ref_name }}-windows-x64.zip
if-no-files-found: error
publish-release:
runs-on: ubuntu-latest
needs:
- upload-linux-artifact
- build-and-upload-windows-package
steps:
- name: Download linux-amd64 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-linux-amd64
path: ./release-files
- name: Download linux-arm64 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-linux-arm64
path: ./release-files
- name: Download linux-armv6 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-linux-armv6
path: ./release-files
- name: Download linux-armv7 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-linux-armv7
path: ./release-files
- name: Download windows-x64 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-${{ github.ref_name }}-windows-x64
path: ./release-files
- name: Publish Release ${{ github.ref_name }}
uses: softprops/action-gh-release@v2
with:
name: ${{ github.ref_name }}
tag_name: ${{ github.ref_name }}
files: ./release-files/*
draft: true
+178
View File
@@ -0,0 +1,178 @@
name: Build Snapshot
on:
push:
branches:
- main
jobs:
build-linux-docker:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.meta.outputs.version }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ secrets.DOCKER_USERNAME }}/ezbookkeeping
tags: |
type=raw,value=SNAPSHOT-{{date 'YYYYMMDD'}}.${{ github.run_id }}
type=raw,value=SNAPSHOT-{{date 'YYYYMMDD'}}
type=raw,value=latest-snapshot
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v8.1.5
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v6
with:
file: Dockerfile
context: .
platforms: |
linux/amd64
linux/arm64/v8
linux/arm/v7
linux/arm/v6
push: true
build-args: |
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
upload-linux-artifact:
needs: build-linux-docker
runs-on: ubuntu-latest
strategy:
matrix:
include:
- arch: linux/amd64
arch_alias: linux-amd64
- arch: linux/arm64/v8
arch_alias: linux-arm64
- arch: linux/arm/v7
arch_alias: linux-armv7
- arch: linux/arm/v6
arch_alias: linux-armv6
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Pull and save packaged files for ${{ matrix.arch }}
run: |
TAG=${{ needs.build-linux-docker.outputs.image-tag }}
IMAGE=${{ secrets.DOCKER_USERNAME }}/ezbookkeeping:${TAG}
docker pull --platform ${{ matrix.arch }} ${IMAGE}
cid=$(docker create "${IMAGE}")
docker cp ${cid}:/ezbookkeeping ./ezbookkeeping
docker rm ${cid}
cd ezbookkeeping
tar -czf ../ezbookkeeping-dev-${{ github.run_id }}-${{ matrix.arch_alias }}.tar.gz *
cd ..
rm -rf ezbookkeeping
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: ezbookkeeping-dev-${{ github.run_id }}-${{ matrix.arch_alias }}
path: ezbookkeeping-dev-${{ github.run_id }}-${{ matrix.arch_alias }}.tar.gz
if-no-files-found: error
build-and-upload-windows-package:
needs: upload-linux-artifact
runs-on: windows-latest
env:
GO_VERSION: "1.25.1"
MINGW_VERSION: "14.2.0"
MINGW_REVISION: "v12-rev2"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Download linux-amd64 packaged files
uses: actions/download-artifact@v4
with:
name: ezbookkeeping-dev-${{ github.run_id }}-linux-amd64
path: artifacts
- name: Extract frontend files from linux-amd64 package
run: |
New-Item -ItemType Directory -Path package
tar -xzf (Get-ChildItem artifacts\ezbookkeeping-dev-${{ github.run_id }}-linux-amd64.tar.gz) -C package
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
- name: Install MinGW
run: |
$mingwVersion = "${{ env.MINGW_VERSION }}"
$mingwRevision = "${{ env.MINGW_REVISION }}"
$url = "https://github.com/niXman/mingw-builds-binaries/releases/download/${mingwVersion}-rt_${mingwRevision}/x86_64-${mingwVersion}-release-posix-seh-ucrt-rt_${mingwRevision}.7z"
$archive = "C:\mingw.7z"
$mingwDir = "C:\mingw64"
Write-Host "Downloading MinGW from ${url}"
Invoke-WebRequest -Uri ${url} -OutFile ${archive}
Remove-Item -Recurse -Force ${mingwDir}
New-Item -ItemType Directory -Path ${mingwDir}
Write-Host "Extracting MinGW to ${mingwDir}"
7z x ${archive} -oC:\
"${mingwDir}\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Build backend for windows-x64
env:
BUILD_PIPELINE: "1"
CHECK_3RD_API: ${{ vars.CHECK_3RD_API }}
SKIP_TESTS: ${{ vars.SKIP_TESTS }}
run: |
.\build.ps1 backend
- name: Package Windows build
run: |
New-Item -ItemType Directory -Path "ezbookkeeping"
New-Item -ItemType Directory -Path "ezbookkeeping\data"
New-Item -ItemType Directory -Path "ezbookkeeping\storage"
New-Item -ItemType Directory -Path "ezbookkeeping\log"
Copy-Item ezbookkeeping.exe -Destination ezbookkeeping\
Copy-Item -Recurse -Force package\public -Destination ezbookkeeping\public
Copy-Item -Recurse -Force conf -Destination ezbookkeeping\conf
Copy-Item -Recurse -Force templates -Destination ezbookkeeping\templates
Copy-Item .\LICENSE -Destination ezbookkeeping\
Push-Location ezbookkeeping
7z a -r -tzip -mx9 ..\ezbookkeeping-dev-${{ github.run_id }}-windows-x64.zip *
Pop-Location
Remove-Item -Recurse -Force ezbookkeeping
- name: Upload Windows artifact
uses: actions/upload-artifact@v4
with:
name: ezbookkeeping-dev-${{ github.run_id }}-windows-x64
path: ezbookkeeping-dev-${{ github.run_id }}-windows-x64.zip
if-no-files-found: error
-57
View File
@@ -1,57 +0,0 @@
name: Docker Release
on:
push:
tags:
- v*
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ secrets.DOCKER_USERNAME }}/ezbookkeeping
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v8.1.5
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v6
with:
file: Dockerfile
context: .
platforms: |
linux/amd64
linux/arm64/v8
linux/arm/v7
linux/arm/v6
push: true
build-args: |
RELEASE_BUILD=1
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
-55
View File
@@ -1,55 +0,0 @@
name: Docker Snapshot
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ secrets.DOCKER_USERNAME }}/ezbookkeeping
tags: |
type=raw,value=SNAPSHOT-{{date 'YYYYMMDD'}}
type=raw,value=latest-snapshot
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v8.1.5
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v6
with:
file: Dockerfile
context: .
platforms: |
linux/amd64
linux/arm64/v8
linux/arm/v7
linux/arm/v6
push: true
build-args: |
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+2 -2
View File
@@ -1,5 +1,5 @@
# Build backend binary file
FROM golang:1.24.5-alpine3.22 AS be-builder
FROM golang:1.25.1-alpine3.22 AS be-builder
ARG RELEASE_BUILD
ARG BUILD_PIPELINE
ARG CHECK_3RD_API
@@ -15,7 +15,7 @@ RUN apk add git gcc g++ libc-dev
RUN ./build.sh backend
# Build frontend files
FROM --platform=$BUILDPLATFORM node:22.18.0-alpine3.22 AS fe-builder
FROM --platform=$BUILDPLATFORM node:24.7.0-alpine3.22 AS fe-builder
ARG RELEASE_BUILD
ARG BUILD_PIPELINE
ENV RELEASE_BUILD=$RELEASE_BUILD
+13 -4
View File
@@ -1,12 +1,14 @@
# ezBookkeeping
[![License](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/mayswind/ezbookkeeping/blob/master/LICENSE)
[![Latest Build](https://img.shields.io/github/actions/workflow/status/mayswind/ezbookkeeping/docker-snapshot.yml?branch=main)](https://github.com/mayswind/ezbookkeeping/actions)
[![Go Report](https://goreportcard.com/badge/github.com/mayswind/ezbookkeeping)](https://goreportcard.com/report/github.com/mayswind/ezbookkeeping)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mayswind/ezbookkeeping)
[![Latest Docker Image Size](https://img.shields.io/docker/image-size/mayswind/ezbookkeeping.svg?style=flat)](https://hub.docker.com/r/mayswind/ezbookkeeping)
[![Latest Release](https://img.shields.io/github/release/mayswind/ezbookkeeping.svg?style=flat)](https://github.com/mayswind/ezbookkeeping/releases)
[![Latest Build](https://img.shields.io/github/actions/workflow/status/mayswind/ezbookkeeping/build-snapshot.yml?branch=main)](https://github.com/mayswind/ezbookkeeping/actions)
[![Latest Docker Image Size](https://img.shields.io/docker/image-size/mayswind/ezbookkeeping.svg?style=flat)](https://hub.docker.com/r/mayswind/ezbookkeeping)
[![Docker Pulls](https://img.shields.io/docker/pulls/mayswind/ezbookkeeping)](https://hub.docker.com/r/mayswind/ezbookkeeping)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mayswind/ezbookkeeping)
[![Recommend By HelloGitHub](https://api.hellogithub.com/v1/widgets/recommend.svg?rid=ded5af09da574ec1811ddb154f1b2093&claim_uid=LT7EZxeBukCnh0K)](https://hellogithub.com/en/repository/mayswind/ezbookkeeping)
[![Trending](https://trendshift.io/api/badge/repositories/12917)](https://trendshift.io/repositories/12917)
## Introduction
ezBookkeeping is a lightweight, self-hosted personal finance app with a user-friendly interface and powerful bookkeeping features. It's easy to deploy, and you can start it with just one single Docker command. Designed to be resource-efficient and highly scalable, it can run smoothly on devices as small as a Raspberry Pi, or scale up to NAS, MicroServers, and even large cluster environments.
@@ -30,6 +32,7 @@ Live Demo: [https://ezbookkeeping-demo.mayswind.net](https://ezbookkeeping-demo.
- PWA support for native-like mobile experience
- Dark mode
- **AI-Powered Features**
- Receipt image recognition
- Supports MCP (Model Context Protocol) for AI integration
- **Powerful Bookkeeping**
- Two-level accounts and categories
@@ -94,6 +97,10 @@ All the files will be packaged in `ezbookkeeping.tar.gz`.
> .\build.bat package -o ezbookkeeping.zip
or
PS > .\build.ps1 package -Output ezbookkeeping.zip
All the files will be packaged in `ezbookkeeping.zip`.
You can also build a Docker image. Make sure you have [Docker](https://www.docker.com/) installed, then follow these steps:
@@ -111,7 +118,7 @@ Want to contribute code? Feel free to fork and send a pull request.
Contributions of all kinds — bug reports, feature suggestions, documentation improvements, or code — are highly appreciated.
Check out our [Contributor Graph](https://github.com/mayswind/ezbookkeeping/graphs/contributors) to see the amazing people whove already helped.
Check out our [Contributor Graph](https://github.com/mayswind/ezbookkeeping/graphs/contributors) to see the amazing people who've already helped.
## Translating
Help make ezBookkeeping accessible to users around the world. If you want to contribute a translation, please refer to our [translation guide](https://ezbookkeeping.mayswind.net/translating).
@@ -123,11 +130,13 @@ Currently available translations:
| de | Deutsch | [@chrgm](https://github.com/chrgm) |
| en | English | / |
| es | Español | [@Miguelonlonlon](https://github.com/Miguelonlonlon) |
| fr | Français | [@brieucdlf](https://github.com/brieucdlf) |
| it | Italiano | [@waron97](https://github.com/waron97) |
| ja | 日本語 | [@tkymmm](https://github.com/tkymmm) |
| nl | Nederlands | [@automagic](https://github.com/automagics) |
| pt-BR | Português (Brasil) | [@thecodergus](https://github.com/thecodergus) |
| ru | Русский | [@artegoser](https://github.com/artegoser) |
| th | ไทย | [@natthavat28](https://github.com/natthavat28) |
| uk | Українська | [@nktlitvinenko](https://github.com/nktlitvinenko) |
| vi | Tiếng Việt | [@f97](https://github.com/f97) |
| zh-Hans | 中文 (简体) | / |
+1 -1
View File
@@ -261,7 +261,7 @@ goto :pre_parse_args
goto :end
)
call 7z a -r -tzip -mx9 ..\%package_file_name% package *
call 7z a -r -tzip -mx9 ..\%package_file_name% *
cd ..
endlocal
+231
View File
@@ -0,0 +1,231 @@
param(
[string]$Type,
[switch]$NoLint,
[switch]$NoTest,
[string]$Output,
[switch]$Release,
[switch]$Help
)
$script:SkipTests = $env:SKIP_TESTS
$script:ReleaseType = "unknown"
$script:Version = ""
$script:CommitHash = ""
$script:BuildUnixTime = ""
$script:BuildDate = ""
function Write-Red($msg) {
Write-Host $msg -ForegroundColor Red
}
function Check-Dependency {
param([string[]]$commands)
foreach ($cmd in $commands) {
if (-not (Get-Command $cmd -ErrorAction SilentlyContinue)) {
Write-Red "Error: `"$cmd`" is required."
exit 127
}
}
}
function Show-Help {
Write-Host "ezBookkeeping build script for Windows PowerShell"
Write-Host ""
Write-Host "Usage:"
Write-Host " build.ps1 type [options]"
Write-Host ""
Write-Host "Types:"
Write-Host " backend Build backend binary file"
Write-Host " frontend Build frontend files"
Write-Host " package Build package archive"
Write-Host ""
Write-Host "Options:"
Write-Host " -Release Build release (The script will use environment variable `"RELEASE_BUILD`" to detect whether this is release building by default)"
Write-Host " -Output <filename> Package file name (for `"package`" type only)"
Write-Host " -NoLint Do not execute lint check before building"
Write-Host " -NoTest Do not execute unit testing before building (You can use environment variable `"SKIP_TESTS`" to skip specified tests)"
Write-Host " -Help Show help"
}
function Parse-Args {
if (-not $Type) {
Show-Help
exit 0
}
if ($Release -or $env:RELEASE_BUILD) {
$script:ReleaseType = "release"
} else {
$script:ReleaseType = "snapshot"
}
}
function Check-Type-Dependencies {
Check-Dependency "git"
switch ($Type.ToLower()) {
"backend" {
Check-Dependency "go","gcc"
}
"frontend" {
Check-Dependency "node","npm"
}
"package" {
Check-Dependency "go","gcc","node","npm","7z"
}
}
}
function Set-Build-Parameters {
$script:Version = (Get-Content package.json | ConvertFrom-Json).version
$script:CommitHash = git rev-parse --short=7 HEAD
$script:BuildUnixTime = [int][double]::Parse((Get-Date -UFormat %s))
$script:BuildDate = Get-Date -Format "yyyyMMdd"
}
function Build-Backend {
Write-Host "Pulling backend dependencies..."
go get .
if (-not $NoLint) {
Write-Host "Executing backend lint checking..."
go vet -v .\...
if ($LASTEXITCODE -ne 0) {
Write-Red "Error: Failed to pass lint checking"
exit 1
}
}
if (-not $NoTest) {
Write-Host "Executing backend unit testing..."
go clean -cache
if (-not $SkipTests) {
go test .\... -v
} else {
Write-Host "(Skip unit test `"$SkipTests`")"
go test .\... -v -skip "$SkipTests"
}
if ($LASTEXITCODE -ne 0) {
Write-Red "Error: Failed to pass unit testing"
exit 1
}
}
$backend_build_extra_arguments = "-X main.Version=$Version "
$backend_build_extra_arguments = "$backend_build_extra_arguments -X main.CommitHash=$CommitHash"
if (-not $Release) {
$backend_build_extra_arguments += " -X main.BuildUnixTime=$BuildUnixTime"
}
Write-Host "Building backend binary file ($ReleaseType)..."
$env:CGO_ENABLED = 1
go build -a -v -trimpath -tags timetzdata -ldflags "-w -s -linkmode external -extldflags '-static' $backend_build_extra_arguments" -o ezbookkeeping.exe ezbookkeeping.go
Remove-Item Env:\CGO_ENABLED -ErrorAction SilentlyContinue
}
function Build-Frontend {
Write-Host "Pulling frontend dependencies..."
npm install
if (-not $NoLint) {
Write-Host "Executing frontend lint checking..."
npm run lint
if ($LASTEXITCODE -ne 0) {
Write-Red "Error: Failed to pass lint checking"
exit 1
}
}
if (-not $NoTest) {
Write-Host "Executing frontend unit testing..."
npm run test
if ($LASTEXITCODE -ne 0) {
Write-Red "Error: Failed to pass unit testing"
exit 1
}
}
Write-Host "Building frontend files ($ReleaseType)..."
if (-not $Release) {
$env:buildUnixTime = $BuildUnixTime
npm run build
Remove-Item Env:\buildUnixTime -ErrorAction SilentlyContinue
} else {
npm run build
}
}
function Build-Package {
$packageFileName = "ezbookkeeping-$Version"
if (-not $Release) {
$packageFileName = "$packageFileName-$BuildDate"
}
$packageFileName = "$packageFileName-windows.zip"
if ($Output) {
$packageFileName = $Output
}
Write-Host "Building package archive '$packageFileName' ($ReleaseType)..."
Build-Backend
Build-Frontend
Remove-Item package -Recurse -Force -ErrorAction SilentlyContinue
New-Item -ItemType Directory -Path "package"
New-Item -ItemType Directory -Path "package\data"
New-Item -ItemType Directory -Path "package\storage"
New-Item -ItemType Directory -Path "package\log"
Copy-Item ezbookkeeping.exe package\
Copy-Item dist package\public -Recurse
Copy-Item conf package\conf -Recurse
Copy-Item templates package\templates -Recurse
Copy-Item LICENSE package\
Push-Location package
7z a -r -tzip -mx9 "..\$packageFileName" *
Pop-Location
}
function Main {
if ($Help) {
Show-Help
exit 0
}
Parse-Args
Check-Type-Dependencies
Set-Build-Parameters
switch ($Type) {
"backend" {
Build-Backend
}
"frontend" {
Build-Frontend
}
"package" {
Build-Package
}
default {
Write-Red "Invalid type: $Type"
Show-Help
exit 2
}
}
}
Main
+16
View File
@@ -9,6 +9,7 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/datastore"
"github.com/mayswind/ezbookkeeping/pkg/duplicatechecker"
"github.com/mayswind/ezbookkeeping/pkg/exchangerates"
"github.com/mayswind/ezbookkeeping/pkg/llm"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/mail"
"github.com/mayswind/ezbookkeeping/pkg/settings"
@@ -90,6 +91,15 @@ func initializeSystem(c *core.CliContext) (*settings.Config, error) {
return nil, err
}
err = llm.InitializeLargeLanguageModelProvider(config)
if err != nil {
if !isDisableBootLog {
log.BootErrorf(c, "[initializer.initializeSystem] initializes large language model provider failed, because %s", err.Error())
}
return nil, err
}
err = uuid.InitializeUuidGenerator(config)
if err != nil {
@@ -162,5 +172,11 @@ func getConfigWithoutSensitiveData(config *settings.Config) *settings.Config {
clonedConfig.WebDAVConfig.Password = "****"
}
if clonedConfig.ReceiptImageRecognitionLLMConfig != nil {
clonedConfig.ReceiptImageRecognitionLLMConfig.OpenAIAPIKey = "****"
clonedConfig.ReceiptImageRecognitionLLMConfig.OpenAICompatibleAPIKey = "****"
clonedConfig.ReceiptImageRecognitionLLMConfig.OpenRouterAPIKey = "****"
}
return clonedConfig
}
+11 -3
View File
@@ -325,6 +325,7 @@ func startWebServer(c *core.CliContext) error {
apiV1Route.GET("/data/statistics.json", bindApi(api.DataManagements.DataStatisticsHandler))
apiV1Route.POST("/data/clear/all.json", bindApi(api.DataManagements.ClearAllDataHandler))
apiV1Route.POST("/data/clear/transactions.json", bindApi(api.DataManagements.ClearAllTransactionsHandler))
apiV1Route.POST("/data/clear/transactions/by_account.json", bindApi(api.DataManagements.ClearAllTransactionsByAccountHandler))
if config.EnableDataExport {
apiV1Route.GET("/data/export.csv", bindCsv(api.DataManagements.ExportDataToEzbookkeepingCSVHandler))
@@ -396,6 +397,13 @@ func startWebServer(c *core.CliContext) error {
apiV1Route.POST("/transaction/templates/move.json", bindApi(api.TransactionTemplates.TemplateMoveHandler))
apiV1Route.POST("/transaction/templates/delete.json", bindApi(api.TransactionTemplates.TemplateDeleteHandler))
// Large Language Models
if config.ReceiptImageRecognitionLLMConfig != nil && config.ReceiptImageRecognitionLLMConfig.LLMProvider != "" {
if config.TransactionFromAIImageRecognition {
apiV1Route.POST("/llm/transactions/recognize_receipt_image.json", bindApi(api.LargeLanguageModels.RecognizeReceiptImageHandler))
}
}
// Exchange Rates
apiV1Route.GET("/exchange_rates/latest.json", bindApi(api.ExchangeRates.LatestExchangeRateHandler))
apiV1Route.POST("/exchange_rates/user_custom/update.json", bindApi(api.ExchangeRates.UserCustomExchangeRateUpdateHandler))
@@ -523,7 +531,7 @@ func bindCachedJs(fn core.DataHandlerFunc, store persistence.CacheStore) gin.Han
if err != nil {
utils.PrintDataErrorResult(c, "text/javascript", err)
} else {
utils.PrintDataSuccessResult(c, "text/javascript", "", result)
utils.PrintDataSuccessResult(c, "text/javascript; charset=utf-8", "", result)
}
})
}
@@ -536,7 +544,7 @@ func bindCsv(fn core.DataHandlerFunc) gin.HandlerFunc {
if err != nil {
utils.PrintDataErrorResult(c, "text/text", err)
} else {
utils.PrintDataSuccessResult(c, "text/csv", fileName, result)
utils.PrintDataSuccessResult(c, "text/csv; charset=utf-8", fileName, result)
}
}
}
@@ -549,7 +557,7 @@ func bindTsv(fn core.DataHandlerFunc) gin.HandlerFunc {
if err != nil {
utils.PrintDataErrorResult(c, "text/text", err)
} else {
utils.PrintDataSuccessResult(c, "text/tab-separated-values", fileName, result)
utils.PrintDataSuccessResult(c, "text/tab-separated-values; charset=utf-8", fileName, result)
}
}
}
+66 -11
View File
@@ -37,6 +37,9 @@ enable_gzip = false
# Set to true to log each request and execution time
log_request = true
# Add X-Request-Id header to response to track user request or error, default is true
request_id_header = true
[mcp]
# Set to true to enable MCP (Model Context Protocol) server (via http / https web server) for AI/LLM access
enable_mcp = false
@@ -161,6 +164,60 @@ webdav_proxy = system
# For "webdav" storage only, set to true to skip tls verification when connect webdav
webdav_skip_tls_verify = false
[llm]
# Set to true to enable creating transactions from AI image recognition results, requires "llm_provider" and its related model id to be configured properly in "llm_image_recognition" section
transaction_from_ai_image_recognition = false
# Maximum allowed AI recognition picture file size (1 - 4294967295 bytes)
max_ai_recognition_picture_size = 10485760
[llm_image_recognition]
# Large Language Model (LLM) provider for receipt image recognition, supports the following types: "openai", "openai_compatible", "openrouter", "ollama", "google_ai"
llm_provider =
# For "openai" llm provider only, OpenAI API secret key, please visit https://platform.openai.com/api-keys for more information
openai_api_key =
# For "openai" llm provider only, receipt image recognition model for creating transactions from images
openai_model_id =
# For "openai_compatible" llm provider only, OpenAI compatible API base url, e.g. "https://api.openai.com/v1/"
openai_compatible_base_url =
# For "openai_compatible" llm provider only, OpenAI compatible API secret key
openai_compatible_api_key =
# For "openai_compatible" llm provider only, receipt image recognition model for creating transactions from images
openai_compatible_model_id =
# For "openrouter" llm provider only, OpenRouter API key, please visit https://openrouter.ai/settings/keys for more information
openrouter_api_key =
# For "openrouter" llm provider only, receipt image recognition model for creating transactions from images
openrouter_model_id =
# For "ollama" llm provider only, Ollama server url, e.g. "http://127.0.0.1:11434/"
ollama_server_url =
# For "ollama" llm provider only, receipt image recognition model for creating transactions from images
ollama_model_id =
# For "google_ai" llm provider only, Google AI Studio API key, please visit https://aistudio.google.com/apikey for more information
google_ai_api_key =
# For "google_ai" llm provider only, receipt image recognition model for creating transactions from images
google_ai_model_id =
# Requesting large language model api timeout (0 - 4294967295 milliseconds)
# Set to 0 to disable timeout for requesting large language model api, default is 60000 (60 seconds)
request_timeout = 60000
# Proxy for ezbookkeeping server requesting large language model api, supports "system" (use system proxy), "none" (do not use proxy), or proxy URL which starts with "http://", "https://" or "socks5://", default is "system"
proxy = system
# Set to true to skip tls verification when request large language model api
skip_tls_verify = false
[uuid]
# Uuid generator type, supports "internal" currently
generator_type = internal
@@ -190,9 +247,6 @@ enable_create_scheduled_transaction = true
# Used for signing, you must change it to keep your user data safe before you first run ezBookkeeping
secret_key =
# Set to true to enable two-factor authorization
enable_two_factor = true
# Token expired seconds (60 - 4294967295), default is 2592000 (30 days)
token_expired_time = 2592000
@@ -215,8 +269,15 @@ max_failures_per_ip_per_minute = 5
# Maximum count of password / token check failures (0 - 4294967295) per user per minute (use the above duplicate checker), default is 5, set to 0 to disable
max_failures_per_user_per_minute = 5
# Add X-Request-Id header to response to track user request or error, default is true
request_id_header = true
[auth]
# Set to true to enable two-factor authorization
enable_two_factor = true
# Set to true to allow users to reset password
enable_forget_password = true
# Set to true to require email must be verified when use forget password
forget_password_require_email_verify = false
[user]
# Set to true to allow users to register account by themselves
@@ -228,12 +289,6 @@ enable_email_verify = false
# Set to true to require email must be verified when login
enable_force_email_verify = false
# Set to true to allow users to reset password
enable_forget_password = true
# Set to true to require email must be verified when use forget password
forget_password_require_email_verify = false
# Set to true to allow users to upload transaction pictures
enable_transaction_picture = true
+9 -9
View File
@@ -1,6 +1,6 @@
module github.com/mayswind/ezbookkeeping
go 1.24
go 1.25
require (
github.com/boombuler/barcode v1.1.0
@@ -8,24 +8,24 @@ require (
github.com/gin-contrib/cache v1.4.1
github.com/gin-contrib/gzip v1.2.3
github.com/gin-gonic/gin v1.10.1
github.com/go-co-op/gocron/v2 v2.16.3
github.com/go-co-op/gocron/v2 v2.16.5
github.com/go-playground/validator/v10 v10.27.0
github.com/go-sql-driver/mysql v1.9.3
github.com/golang-jwt/jwt/v5 v5.3.0
github.com/invopop/jsonschema v0.13.0
github.com/lib/pq v1.10.9
github.com/mattn/go-sqlite3 v1.14.30
github.com/mattn/go-sqlite3 v1.14.32
github.com/minio/minio-go/v7 v7.0.95
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pquerna/otp v1.5.0
github.com/sirupsen/logrus v1.9.3
github.com/stretchr/testify v1.10.0
github.com/urfave/cli/v3 v3.3.8
github.com/stretchr/testify v1.11.1
github.com/urfave/cli/v3 v3.4.1
github.com/wk8/go-ordered-map/v2 v2.1.8
github.com/xuri/excelize/v2 v2.9.0
golang.org/x/crypto v0.40.0
golang.org/x/net v0.42.0
golang.org/x/text v0.27.0
golang.org/x/crypto v0.41.0
golang.org/x/net v0.43.0
golang.org/x/text v0.28.0
gopkg.in/ini.v1 v1.67.0
gopkg.in/mail.v2 v2.3.1
xorm.io/builder v0.3.13
@@ -91,7 +91,7 @@ require (
github.com/xuri/nfp v0.0.1 // indirect
golang.org/x/arch v0.18.0 // indirect
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/sys v0.35.0 // indirect
google.golang.org/protobuf v1.36.6 // indirect
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
+16 -16
View File
@@ -50,8 +50,8 @@ github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.10.1 h1:T0ujvqyCSqRopADpgPgiTT63DUQVSfojyME59Ei63pQ=
github.com/gin-gonic/gin v1.10.1/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
github.com/go-co-op/gocron/v2 v2.16.3 h1:kYqukZqBa8RC2+AFAHnunmKcs9GRTjwBo8WRF3I6cbI=
github.com/go-co-op/gocron/v2 v2.16.3/go.mod h1:aTf7/+5Jo2E+cyAqq625UQ6DzpkV96b22VHIUAt6l3c=
github.com/go-co-op/gocron/v2 v2.16.5 h1:j228Jxk7bb9CF8LKR3gS+bK3rcjRUINjlVI+ZMp26Ss=
github.com/go-co-op/gocron/v2 v2.16.5/go.mod h1:zAfC/GFQ668qHxOVl/D68Jh5Ce7sDqX6TJnSQyRkRBc=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
@@ -101,8 +101,8 @@ github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.30 h1:bVreufq3EAIG1Quvws73du3/QgdeZ3myglJlrzSYYCY=
github.com/mattn/go-sqlite3 v1.14.30/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mattn/go-sqlite3 v1.14.32 h1:JD12Ag3oLy1zQA+BNn74xRgaBbdhbNIDYvQUEuuErjs=
github.com/mattn/go-sqlite3 v1.14.32/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/memcachier/mc/v3 v3.0.3 h1:qii+lDiPKi36O4Xg+HVKwHu6Oq+Gt17b+uEiA0Drwv4=
github.com/memcachier/mc/v3 v3.0.3/go.mod h1:GzjocBahcXPxt2cmqzknrgqCOmMxiSzhVKPOe90Tpug=
github.com/minio/crc64nvme v1.0.2 h1:6uO1UxGAD+kwqWWp7mBFsi5gAse66C4NXO8cmcVculg=
@@ -153,8 +153,8 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=
github.com/tealeg/xlsx v1.0.5 h1:+f8oFmvY8Gw1iUXzPk+kz+4GpbDZPK1FhPiQRd+ypgE=
@@ -166,8 +166,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.14 h1:yOQvXCBc3Ij46LRkRoh4Yd5qK6LVOgi0bYOXfb7ifjw=
github.com/ugorji/go/codec v1.2.14/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/urfave/cli/v3 v3.3.8 h1:BzolUExliMdet9NlJ/u4m5vHSotJ3PzEqSAZ1oPMa/E=
github.com/urfave/cli/v3 v3.3.8/go.mod h1:FJSKtM/9AiiTOJL4fJ6TbMUkxBXn7GO9guZqoZtpYpo=
github.com/urfave/cli/v3 v3.4.1 h1:1M9UOCy5bLmGnuu1yn3t3CB4rG79Rtoxuv1sPhnm6qM=
github.com/urfave/cli/v3 v3.4.1/go.mod h1:FJSKtM/9AiiTOJL4fJ6TbMUkxBXn7GO9guZqoZtpYpo=
github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc=
github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
@@ -180,21 +180,21 @@ github.com/xuri/nfp v0.0.1/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.18.0 h1:WN9poc33zL4AzGxqf8VtpKUnGvMi8O9lhNyBMF/85qc=
golang.org/x/arch v0.18.0/go.mod h1:bdwinDaKcfZUGpH09BB7ZmOfhalA8lQdzl62l8gGWsk=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4=
golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc=
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8/go.mod h1:jj3sYF3dwk5D+ghuXyeI3r5MFf+NT2An6/9dOA95KSI=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc h1:2gGKlE2+asNV9m7xrywl36YYNnBG5ZQ0r/BOOxqPpmk=
+2452 -1473
View File
File diff suppressed because it is too large Load Diff
+26 -24
View File
@@ -1,6 +1,6 @@
{
"name": "ezbookkeeping",
"version": "1.0.0",
"version": "1.1.0",
"private": true,
"repository": {
"type": "git",
@@ -30,6 +30,7 @@
"framework7": "^8.3.4",
"framework7-icons": "^5.0.5",
"framework7-vue": "^8.3.4",
"jalaali-js": "^1.2.8",
"leaflet": "^1.9.4",
"line-awesome": "^1.3.0",
"moment": "^2.30.1",
@@ -39,41 +40,42 @@
"skeleton-elements": "^4.0.1",
"swiper": "^10.2.0",
"ua-parser-js": "^1.0.39",
"vue": "^3.5.18",
"vue": "^3.5.21",
"vue-echarts": "^7.0.3",
"vue-i18n": "^11.1.11",
"vue-i18n": "^11.1.12",
"vue-router": "^4.5.1",
"vue3-perfect-scrollbar": "^2.0.0",
"vuedraggable": "^4.1.0",
"vuetify": "^3.9.3"
"vuetify": "^3.9.7"
},
"devDependencies": {
"@jest/globals": "^29.7.0",
"@tsconfig/node22": "^22.0.2",
"@jest/globals": "^30.1.2",
"@tsconfig/node24": "^24.0.1",
"@types/cbor-js": "^0.1.1",
"@types/crypto-js": "^4.2.2",
"@types/git-rev-sync": "^2.0.2",
"@types/jest": "^29.5.14",
"@types/node": "^22.15.29",
"@types/jalaali-js": "^1.2.0",
"@types/jest": "^30.0.0",
"@types/node": "^24.3.1",
"@types/ua-parser-js": "^0.7.39",
"@vitejs/plugin-vue": "^5.2.4",
"@vue/eslint-config-typescript": "^14.5.0",
"@vue/tsconfig": "^0.7.0",
"cross-env": "^7.0.3",
"eslint": "^9.28.0",
"eslint-plugin-vue": "^10.1.0",
"@vitejs/plugin-vue": "^6.0.1",
"@vue/eslint-config-typescript": "^14.6.0",
"@vue/tsconfig": "^0.8.1",
"cross-env": "^10.0.0",
"eslint": "^9.35.0",
"eslint-plugin-vue": "^10.4.0",
"git-rev-sync": "^3.0.2",
"jest": "^29.7.0",
"postcss-preset-env": "^10.2.0",
"sass": "^1.89.1",
"ts-jest": "^29.3.4",
"jest": "^30.1.3",
"postcss-preset-env": "^10.3.1",
"sass": "^1.92.1",
"ts-jest": "^29.4.1",
"ts-node": "^10.9.2",
"typescript": "^5.8.3",
"vite": "^6.3.5",
"vite-plugin-checker": "^0.9.3",
"vite-plugin-pwa": "^1.0.0",
"vite-plugin-vuetify": "^2.1.1",
"vue-tsc": "^2.2.10"
"typescript": "^5.9.2",
"vite": "^7.1.4",
"vite-plugin-checker": "^0.10.3",
"vite-plugin-pwa": "^1.0.3",
"vite-plugin-vuetify": "^2.1.2",
"vue-tsc": "^3.0.6"
},
"browserslist": [
"last 5 Chrome versions",
+56
View File
@@ -15,6 +15,7 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const pageCountForClearTransactions = 1000
const pageCountForDataExport = 1000
// DataManagementsApi represents data management api
@@ -232,6 +233,61 @@ func (a *DataManagementsApi) ClearAllTransactionsHandler(c *core.WebContext) (an
return true, nil
}
// ClearAllTransactionsByAccountHandler deletes all transactions of specified account
func (a *DataManagementsApi) ClearAllTransactionsByAccountHandler(c *core.WebContext) (any, *errs.Error) {
var clearDataReq models.ClearAccountTransactionsRequest
err := c.ShouldBindJSON(&clearDataReq)
if err != nil {
log.Warnf(c, "[data_managements.ClearAllTransactionsByAccountHandler] parse request failed, because %s", err.Error())
return nil, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
if !errs.IsCustomError(err) {
log.Warnf(c, "[data_managements.ClearAllTransactionsByAccountHandler] failed to get user for user \"uid:%d\", because %s", uid, err.Error())
}
return nil, errs.ErrUserNotFound
}
if !a.users.IsPasswordEqualsUserPassword(clearDataReq.Password, user) {
return nil, errs.ErrUserPasswordWrong
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_CLEAR_ALL_DATA) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
account, err := a.accounts.GetAccountByAccountId(c, uid, clearDataReq.AccountId)
if err != nil {
log.Errorf(c, "[data_managements.ClearAllTransactionsByAccountHandler] failed to get account \"id:%d\" for user \"uid:%d\", because %s", uid, clearDataReq.AccountId, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
if account.Hidden {
return nil, errs.ErrCannotDeleteTransactionInHiddenAccount
}
if account.Type == models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS {
return nil, errs.ErrCannotDeleteTransactionInParentAccount
}
err = a.transactions.DeleteAllTransactionsOfAccount(c, uid, account.AccountId, pageCountForClearTransactions)
if err != nil {
log.Errorf(c, "[data_managements.ClearAllTransactionsByAccountHandler] failed to delete all transactions in account \"id:%d\", because %s", account.AccountId, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
log.Infof(c, "[data_managements.ClearAllTransactionsByAccountHandler] user \"uid:%d\" has cleared all transactions in account \"id:%d\"", uid, account.AccountId)
return true, nil
}
func (a *DataManagementsApi) getExportedFileContent(c *core.WebContext, fileType string) ([]byte, string, *errs.Error) {
if !a.CurrentConfig().EnableDataExport {
return nil, "", errs.ErrDataExportNotAllowed
+374
View File
@@ -0,0 +1,374 @@
package api
import (
"bytes"
"encoding/json"
"io"
"strings"
"time"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/services"
"github.com/mayswind/ezbookkeeping/pkg/settings"
"github.com/mayswind/ezbookkeeping/pkg/templates"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
// LargeLanguageModelsApi represents large language models api
type LargeLanguageModelsApi struct {
ApiUsingConfig
transactionCategories *services.TransactionCategoryService
transactionTags *services.TransactionTagService
accounts *services.AccountService
users *services.UserService
}
// Initialize a large language models api singleton instance
var (
LargeLanguageModels = &LargeLanguageModelsApi{
ApiUsingConfig: ApiUsingConfig{
container: settings.Container,
},
transactionCategories: services.TransactionCategories,
transactionTags: services.TransactionTags,
accounts: services.Accounts,
users: services.Users,
}
)
// RecognizeReceiptImageHandler returns the recognized receipt image result
func (a *LargeLanguageModelsApi) RecognizeReceiptImageHandler(c *core.WebContext) (any, *errs.Error) {
if a.CurrentConfig().ReceiptImageRecognitionLLMConfig == nil || a.CurrentConfig().ReceiptImageRecognitionLLMConfig.LLMProvider == "" || !a.CurrentConfig().TransactionFromAIImageRecognition {
return nil, errs.ErrLargeLanguageModelProviderNotEnabled
}
utcOffset, err := c.GetClientTimezoneOffset()
if err != nil {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] cannot get client timezone offset, because %s", err.Error())
return nil, errs.ErrClientTimezoneOffsetInvalid
}
timezone := time.FixedZone("Client Timezone", int(utcOffset)*60)
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
if !errs.IsCustomError(err) {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get user for user \"uid:%d\", because %s", uid, err.Error())
}
return false, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_CREATE_TRANSACTION_FROM_AI_IMAGE_RECOGNITION) {
return false, errs.ErrNotPermittedToPerformThisAction
}
form, err := c.MultipartForm()
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get multi-part form data for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrParameterInvalid
}
imageFiles := form.File["image"]
if len(imageFiles) < 1 {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] there is no image in request for user \"uid:%d\"", uid)
return nil, errs.ErrNoAIRecognitionImage
}
if imageFiles[0].Size < 1 {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] the size of image in request is zero for user \"uid:%d\"", uid)
return nil, errs.ErrAIRecognitionImageIsEmpty
}
if imageFiles[0].Size > int64(a.CurrentConfig().MaxAIRecognitionPictureFileSize) {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] the upload file size \"%d\" exceeds the maximum size \"%d\" of image for user \"uid:%d\"", imageFiles[0].Size, a.CurrentConfig().MaxAIRecognitionPictureFileSize, uid)
return nil, errs.ErrExceedMaxAIRecognitionImageFileSize
}
fileExtension := utils.GetFileNameExtension(imageFiles[0].Filename)
contentType := utils.GetImageContentType(fileExtension)
if contentType == "" {
log.Warnf(c, "[large_language_models.RecognizeReceiptImageHandler] the file extension \"%s\" of image in request is not supported for user \"uid:%d\"", fileExtension, uid)
return nil, errs.ErrImageTypeNotSupported
}
imageFile, err := imageFiles[0].Open()
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get image file from request for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrOperationFailed
}
defer imageFile.Close()
imageData, err := io.ReadAll(imageFile)
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to read image file from request for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrOperationFailed
}
accounts, err := a.accounts.GetAllAccountsByUid(c, uid)
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get all accounts for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
accountMap := a.accounts.GetVisibleAccountNameMapByList(accounts)
accountNames := make([]string, 0, len(accounts))
for i := 0; i < len(accounts); i++ {
if accounts[i].Hidden || accounts[i].Type == models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS {
continue
}
accountNames = append(accountNames, accounts[i].Name)
}
categories, err := a.transactionCategories.GetAllCategoriesByUid(c, uid, 0, -1)
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get categories for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
incomeCategoryMap := make(map[string]*models.TransactionCategory)
incomeCategoryNames := make([]string, 0)
expenseCategoryMap := make(map[string]*models.TransactionCategory)
expenseCategoryNames := make([]string, 0)
transferCategoryMap := make(map[string]*models.TransactionCategory)
transferCategoryNames := make([]string, 0)
for i := 0; i < len(categories); i++ {
category := categories[i]
if category.Hidden || category.ParentCategoryId == models.LevelOneTransactionCategoryParentId {
continue
}
if category.Type == models.CATEGORY_TYPE_INCOME {
incomeCategoryMap[category.Name] = category
incomeCategoryNames = append(incomeCategoryNames, category.Name)
} else if category.Type == models.CATEGORY_TYPE_EXPENSE {
expenseCategoryMap[category.Name] = category
expenseCategoryNames = append(expenseCategoryNames, category.Name)
} else if category.Type == models.CATEGORY_TYPE_TRANSFER {
transferCategoryMap[category.Name] = category
transferCategoryNames = append(transferCategoryNames, category.Name)
}
}
tags, err := a.transactionTags.GetAllTagsByUid(c, uid)
if err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to get tags for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
tagMap := a.transactionTags.GetVisibleTagNameMapByList(tags)
tagNames := make([]string, 0, len(tags))
for i := 0; i < len(tags); i++ {
if tags[i].Hidden {
continue
}
tagNames = append(tagNames, tags[i].Name)
}
systemPrompt, err := templates.GetTemplate(templates.SYSTEM_PROMPT_RECEIPT_IMAGE_RECOGNITION)
if err != nil {
return nil, errs.Or(err, errs.ErrOperationFailed)
}
systemPromptParams := map[string]any{
"CurrentDateTime": utils.FormatUnixTimeToLongDateTime(time.Now().Unix(), timezone),
"AllExpenseCategoryNames": strings.Join(expenseCategoryNames, "\n"),
"AllIncomeCategoryNames": strings.Join(incomeCategoryNames, "\n"),
"AllTransferCategoryNames": strings.Join(transferCategoryNames, "\n"),
"AllAccountNames": strings.Join(accountNames, "\n"),
"AllTagNames": strings.Join(tagNames, "\n"),
}
var bodyBuffer bytes.Buffer
err = systemPrompt.Execute(&bodyBuffer, systemPromptParams)
if err != nil {
return nil, errs.Or(err, errs.ErrOperationFailed)
}
llmRequest := &data.LargeLanguageModelRequest{
Stream: false,
SystemPrompt: strings.ReplaceAll(bodyBuffer.String(), "\r\n", "\n"),
UserPrompt: imageData,
UserPromptType: data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL,
UserPromptContentType: contentType,
}
llmResponse, err := llm.Container.GetJsonResponseByReceiptImageRecognitionModel(c, c.GetCurrentUid(), a.CurrentConfig(), llmRequest)
if err != nil {
return nil, errs.Or(err, errs.ErrOperationFailed)
}
if llmResponse == nil || len(llmResponse.Content) == 0 || strings.HasPrefix(llmResponse.Content, "{}") {
return nil, errs.ErrNoTransactionInformationInImage
}
var result *models.RecognizedReceiptImageResult
if err := json.Unmarshal([]byte(llmResponse.Content), &result); err != nil {
log.Errorf(c, "[large_language_models.RecognizeReceiptImageHandler] failed to unmarshal recognized receipt image result from llm response \"%s\" for user \"uid:%d\", because %s", llmResponse.Content, uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
return a.parseRecognizedReceiptImageResponse(c, uid, utcOffset, result, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
func (a *LargeLanguageModelsApi) parseRecognizedReceiptImageResponse(c *core.WebContext, uid int64, utcOffset int16, recognizedResult *models.RecognizedReceiptImageResult, accountMap map[string]*models.Account, expenseCategoryMap map[string]*models.TransactionCategory, incomeCategoryMap map[string]*models.TransactionCategory, transferCategoryMap map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (*models.RecognizedReceiptImageResponse, *errs.Error) {
recognizedReceiptImageResponse := &models.RecognizedReceiptImageResponse{
Type: models.TRANSACTION_TYPE_EXPENSE,
}
if recognizedResult == nil {
log.Errorf(c, "[large_language_models.parseRecognizedReceiptImageResponse] recoginzed result is null")
return nil, errs.ErrNoTransactionInformationInImage
}
if recognizedResult.Type == "income" {
recognizedReceiptImageResponse.Type = models.TRANSACTION_TYPE_INCOME
if len(recognizedResult.CategoryName) > 0 {
category, exists := incomeCategoryMap[recognizedResult.CategoryName]
if exists {
recognizedReceiptImageResponse.CategoryId = category.CategoryId
}
}
} else if recognizedResult.Type == "expense" {
recognizedReceiptImageResponse.Type = models.TRANSACTION_TYPE_EXPENSE
if len(recognizedResult.CategoryName) > 0 {
category, exists := expenseCategoryMap[recognizedResult.CategoryName]
if exists {
recognizedReceiptImageResponse.CategoryId = category.CategoryId
}
}
} else if recognizedResult.Type == "transfer" {
recognizedReceiptImageResponse.Type = models.TRANSACTION_TYPE_TRANSFER
if len(recognizedResult.CategoryName) > 0 {
category, exists := transferCategoryMap[recognizedResult.CategoryName]
if exists {
recognizedReceiptImageResponse.CategoryId = category.CategoryId
}
}
} else if len(recognizedResult.Type) == 0 {
return nil, errs.ErrNoTransactionInformationInImage
} else {
log.Errorf(c, "[large_language_models.parseRecognizedReceiptImageResponse] recoginzed transaction type \"%s\" is invalid", recognizedResult.Type)
return nil, errs.ErrOperationFailed
}
if len(recognizedResult.Time) > 0 {
longDateTime := a.getLongDateTime(recognizedResult.Time)
timestamp, err := utils.ParseFromLongDateTime(longDateTime, utcOffset)
if err != nil {
log.Warnf(c, "[large_language_models.parseRecognizedReceiptImageResponse] recoginzed time \"%s\" is invalid", recognizedResult.Time)
} else {
recognizedReceiptImageResponse.Time = timestamp.Unix()
}
}
if len(recognizedResult.Amount) > 0 {
amount, err := utils.ParseAmount(recognizedResult.Amount)
if err != nil {
log.Errorf(c, "[large_language_models.parseRecognizedReceiptImageResponse] recoginzed amount \"%s\" is invalid", recognizedResult.Amount)
return nil, errs.ErrOperationFailed
}
recognizedReceiptImageResponse.SourceAmount = amount
if recognizedReceiptImageResponse.Type == models.TRANSACTION_TYPE_TRANSFER && len(recognizedResult.DestinationAmount) > 0 {
destinationAmount, err := utils.ParseAmount(recognizedResult.DestinationAmount)
if err != nil {
log.Errorf(c, "[large_language_models.parseRecognizedReceiptImageResponse] recoginzed destination amount \"%s\" is invalid", recognizedResult.DestinationAmount)
return nil, errs.ErrOperationFailed
}
recognizedReceiptImageResponse.DestinationAmount = destinationAmount
}
}
if len(recognizedResult.AccountName) > 0 {
account, exists := accountMap[recognizedResult.AccountName]
if exists {
recognizedReceiptImageResponse.SourceAccountId = account.AccountId
}
}
if len(recognizedResult.DestinationAccountName) > 0 {
account, exists := accountMap[recognizedResult.DestinationAccountName]
if exists {
recognizedReceiptImageResponse.DestinationAccountId = account.AccountId
}
}
if len(recognizedResult.TagNames) > 0 {
tagIds := make([]string, 0, len(recognizedResult.TagNames))
for i := 0; i < len(recognizedResult.TagNames); i++ {
tagName := recognizedResult.TagNames[i]
tag, exists := tagMap[tagName]
if exists {
tagIds = append(tagIds, utils.Int64ToString(tag.TagId))
}
}
recognizedReceiptImageResponse.TagIds = tagIds
}
if len(recognizedResult.Description) > 0 {
recognizedReceiptImageResponse.Comment = recognizedResult.Description
}
return recognizedReceiptImageResponse, nil
}
func (a *LargeLanguageModelsApi) getLongDateTime(dateTime string) string {
if utils.IsValidLongDateTimeFormat(dateTime) {
return dateTime
}
if utils.IsValidLongDateTimeWithoutSecondFormat(dateTime) {
return dateTime + ":00"
}
if utils.IsValidLongDateFormat(dateTime) {
return dateTime + " 00:00:00"
}
return dateTime
}
+1 -3
View File
@@ -3,8 +3,6 @@ package api
import (
"encoding/json"
"github.com/gin-gonic/gin"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
@@ -233,7 +231,7 @@ func (a *ModelContextProtocolAPI) CallToolHandler(c *core.WebContext, jsonRPCReq
// PingHandler return the ping response for model context protocol
func (a *ModelContextProtocolAPI) PingHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
return gin.H{}, nil
return core.O{}, nil
}
// GetTransactionService implements the MCPAvailableServices interface
+6
View File
@@ -47,6 +47,12 @@ func (a *ServerSettingsApi) ServerSettingsJavascriptHandler(c *core.WebContext)
a.appendBooleanSetting(builder, "mcp", config.EnableMCPServer)
}
if config.ReceiptImageRecognitionLLMConfig != nil && config.ReceiptImageRecognitionLLMConfig.LLMProvider != "" {
if config.TransactionFromAIImageRecognition {
a.appendBooleanSetting(builder, "llmt", config.TransactionFromAIImageRecognition)
}
}
if config.LoginPageTips.Enabled {
a.appendMultiLanguageTipSetting(builder, "lpt", config.LoginPageTips)
}
+21 -2
View File
@@ -549,6 +549,25 @@ func (a *TransactionsApi) TransactionAmountsHandler(c *core.WebContext) (any, *e
return nil, errs.ErrQueryItemsTooMuch
}
excludeAccountIds := make([]int64, 0)
excludeCategoryIds := make([]int64, 0)
if transactionAmountsReq.ExcludeAccountIds != "" {
excludeAccountIds, err = utils.StringArrayToInt64Array(strings.Split(transactionAmountsReq.ExcludeAccountIds, ","))
if err != nil {
return nil, errs.ErrAccountIdInvalid
}
}
if transactionAmountsReq.ExcludeCategoryIds != "" {
excludeCategoryIds, err = utils.StringArrayToInt64Array(strings.Split(transactionAmountsReq.ExcludeCategoryIds, ","))
if err != nil {
return nil, errs.ErrTransactionCategoryIdInvalid
}
}
utcOffset, err := c.GetClientTimezoneOffset()
if err != nil {
@@ -571,7 +590,7 @@ func (a *TransactionsApi) TransactionAmountsHandler(c *core.WebContext) (any, *e
for i := 0; i < len(requestItems); i++ {
requestItem := requestItems[i]
incomeAmounts, expenseAmounts, err := a.transactions.GetAccountsTotalIncomeAndExpense(c, uid, requestItem.StartTime, requestItem.EndTime, utcOffset, transactionAmountsReq.UseTransactionTimezone)
incomeAmounts, expenseAmounts, err := a.transactions.GetAccountsTotalIncomeAndExpense(c, uid, requestItem.StartTime, requestItem.EndTime, excludeAccountIds, excludeCategoryIds, utcOffset, transactionAmountsReq.UseTransactionTimezone)
if err != nil {
log.Errorf(c, "[transactions.TransactionAmountsHandler] failed to get transaction amounts item for user \"uid:%d\", because %s", uid, err.Error())
@@ -1428,7 +1447,7 @@ func (a *TransactionsApi) TransactionParseImportFileHandler(c *core.WebContext)
return nil, errs.Or(err, errs.ErrOperationFailed)
}
tagMap := a.transactionTags.GetTagNameMapByList(tags)
tagMap := a.transactionTags.GetVisibleTagNameMapByList(tags)
parsedTransactions, _, _, _, _, _, err := dataImporter.ParseImportedData(c, user, fileData, utcOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
+1 -1
View File
@@ -957,7 +957,7 @@ func (l *UserDataCli) getUserEssentialDataForImport(c *core.CliContext, uid int6
return nil, nil, nil, nil, nil, err
}
tagMap = l.tags.GetTagNameMapByList(tags)
tagMap = l.tags.GetVisibleTagNameMapByList(tags)
return accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap, nil
}
@@ -102,6 +102,7 @@ func TestAlipayCsvFileImporterParseImportedData_ParseRefundTransaction(t *testin
DefaultCurrency: "CNY",
}
// refund
data1, err := simplifiedchinese.GB18030.NewEncoder().String("支付宝交易记录明细查询\n" +
"账号:[xxx@xxx.xxx]\n" +
"起始日期:[2024-01-01 00:00:00] 终止日期:[2024-09-01 23:59:59]\n" +
@@ -121,6 +122,7 @@ func TestAlipayCsvFileImporterParseImportedData_ParseRefundTransaction(t *testin
assert.Equal(t, "", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "", allNewTransactions[0].OriginalCategoryName)
// tax refund
data2, err := simplifiedchinese.GB18030.NewEncoder().String("支付宝交易记录明细查询\n" +
"账号:[xxx@xxx.xxx]\n" +
"起始日期:[2024-01-01 00:00:00] 终止日期:[2024-09-01 23:59:59]\n" +
@@ -141,6 +143,46 @@ func TestAlipayCsvFileImporterParseImportedData_ParseRefundTransaction(t *testin
assert.Equal(t, "", allNewTransactions[0].OriginalCategoryName)
}
func TestAlipayCsvFileImporterParseImportedData_ParseInvestmentRefundTransaction(t *testing.T) {
converter := AlipayAppTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data1, err := simplifiedchinese.GB18030.NewEncoder().String("------------------------------------------------------------------------------------\n" +
"导出信息:\n" +
"姓名:xxx\n" +
"支付宝账户:xxx@xxx.xxx\n" +
"起始时间:[2024-01-01 00:00:00] 终止时间:[2024-09-01 23:59:59]\n" +
"导出交易类型:[全部]\n" +
"------------------------支付宝(中国)网络技术有限公司 电子客户回单------------------------\n" +
"交易时间,交易对方,商品说明,收/支,金额,收/付款方式,交易状态,\n" +
"2024-09-01 01:00:00,Test Account2,xxx-买入,不计收支,0.01,Test Account,退款成功,\n" +
"2024-09-01 02:00:00,Test Account2,xxx-买入退款,不计收支,0.01,Test Account,退款成功,\n")
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 2, len(allNewTransactions))
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, "2024-09-01 01:00:00", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime), time.UTC))
assert.Equal(t, int64(1), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[1].Type)
assert.Equal(t, "2024-09-01 02:00:00", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime), time.UTC))
assert.Equal(t, int64(1), allNewTransactions[1].Amount)
assert.Equal(t, "Test Account2", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "Test Account", allNewTransactions[1].OriginalDestinationAccountName)
}
func TestAlipayCsvFileImporterParseImportedData_ParseInvalidTime(t *testing.T) {
converter := AlipayWebTransactionDataCsvFileImporter
context := core.NewNullContext()
@@ -380,38 +422,110 @@ func TestAlipayCsvFileImporterParseImportedData_ParseRelatedAccount(t *testing.T
"起始时间:[2024-01-01 00:00:00] 终止时间:[2024-09-01 23:59:59]\n" +
"导出交易类型:[全部]\n" +
"------------------------支付宝(中国)网络技术有限公司 电子客户回单------------------------\n" +
"交易时间,商品说明,收/支,金额,收/付款方式,交易状态,\n" +
"2024-09-01 03:45:07,余额宝-单次转入,不计收支,0.01,Test Account,交易成功,\n" +
"2024-09-01 05:07:29,信用卡还款,不计收支,0.02,Test Account2,交易成功,\n")
"交易时间,交易对方,商品说明,收/支,金额,收/付款方式,交易状态,备注,\n" +
"2024-09-01 00:00:00,xxx,xxx-收益发放,不计收支,0.01,Test Account,交易成功,earning,\n" +
"2024-09-01 01:00:00,Test Account2,xxx-买入,不计收支,0.01,Test Account,交易成功,purchase investment,\n" +
"2024-09-01 02:00:00,Test Account2,xxx-卖出至xxx,不计收支,0.01,Test Account,交易成功,sell investment,\n" +
"2024-09-01 03:00:00,xxx,充值-普通充值,不计收支,0.01,Test Account,交易成功,transfer to alipay wallet,\n" +
"2024-09-01 04:00:00,Test Account3,提现-实时提现,不计收支,0.01,Test Account,交易成功,transfer from alipay wallet,\n" +
"2024-09-01 05:00:00,Test Account3,xxx-单次转入,不计收支,0.01,Test Account,交易成功,transfer in,\n" +
"2024-09-01 06:00:00,Test Account3,xxx-转出到银行卡,不计收支,0.01,Test Account,交易成功,transfer out,\n" +
"2024-09-01 07:00:00,Test Account3,转账xxx,不计收支,0.01,Test Account,交易成功,transfer,\n" +
"2024-09-01 08:00:00,Test Account4,信用卡还款,不计收支,0.01,Test Account,还款成功,repayment,\n")
assert.Nil(t, err)
allNewTransactions, allNewAccounts, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 2, len(allNewTransactions))
assert.Equal(t, 3, len(allNewAccounts))
assert.Equal(t, 9, len(allNewTransactions))
assert.Equal(t, 6, len(allNewAccounts))
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, int64(1), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, "earning", allNewTransactions[0].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[1].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, int64(2), allNewTransactions[1].Amount)
assert.Equal(t, "Test Account2", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "", allNewTransactions[1].OriginalDestinationAccountName)
assert.Equal(t, int64(1), allNewTransactions[1].Amount)
assert.Equal(t, "Test Account", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[1].OriginalDestinationAccountName)
assert.Equal(t, "purchase investment", allNewTransactions[1].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[2].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[2].Uid)
assert.Equal(t, int64(1), allNewTransactions[2].Amount)
assert.Equal(t, "Test Account2", allNewTransactions[2].OriginalSourceAccountName)
assert.Equal(t, "Test Account", allNewTransactions[2].OriginalDestinationAccountName)
assert.Equal(t, "sell investment", allNewTransactions[2].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[3].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[3].Uid)
assert.Equal(t, int64(1), allNewTransactions[3].Amount)
assert.Equal(t, "", allNewTransactions[3].OriginalSourceAccountName)
assert.Equal(t, "Alipay", allNewTransactions[3].OriginalDestinationAccountName)
assert.Equal(t, "transfer to alipay wallet", allNewTransactions[3].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[4].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[4].Uid)
assert.Equal(t, int64(1), allNewTransactions[4].Amount)
assert.Equal(t, "Alipay", allNewTransactions[4].OriginalSourceAccountName)
assert.Equal(t, "Test Account3", allNewTransactions[4].OriginalDestinationAccountName)
assert.Equal(t, "transfer from alipay wallet", allNewTransactions[4].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[5].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[5].Uid)
assert.Equal(t, int64(1), allNewTransactions[5].Amount)
assert.Equal(t, "Test Account", allNewTransactions[5].OriginalSourceAccountName)
assert.Equal(t, "Test Account3", allNewTransactions[5].OriginalDestinationAccountName)
assert.Equal(t, "transfer in", allNewTransactions[5].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[6].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[6].Uid)
assert.Equal(t, int64(1), allNewTransactions[6].Amount)
assert.Equal(t, "Test Account", allNewTransactions[6].OriginalSourceAccountName)
assert.Equal(t, "Test Account3", allNewTransactions[6].OriginalDestinationAccountName)
assert.Equal(t, "transfer out", allNewTransactions[6].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[7].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[7].Uid)
assert.Equal(t, int64(1), allNewTransactions[7].Amount)
assert.Equal(t, "Test Account", allNewTransactions[7].OriginalSourceAccountName)
assert.Equal(t, "Test Account3", allNewTransactions[7].OriginalDestinationAccountName)
assert.Equal(t, "transfer", allNewTransactions[7].Comment)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[8].Type)
assert.Equal(t, int64(1234567890), allNewTransactions[8].Uid)
assert.Equal(t, int64(1), allNewTransactions[8].Amount)
assert.Equal(t, "Test Account", allNewTransactions[8].OriginalSourceAccountName)
assert.Equal(t, "Test Account4", allNewTransactions[8].OriginalDestinationAccountName)
assert.Equal(t, "repayment", allNewTransactions[8].Comment)
assert.Equal(t, int64(1234567890), allNewAccounts[0].Uid)
assert.Equal(t, "Test Account", allNewAccounts[0].Name)
assert.Equal(t, "CNY", allNewAccounts[0].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[1].Uid)
assert.Equal(t, "", allNewAccounts[1].Name)
assert.Equal(t, "Test Account2", allNewAccounts[1].Name)
assert.Equal(t, "CNY", allNewAccounts[1].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[2].Uid)
assert.Equal(t, "Test Account2", allNewAccounts[2].Name)
assert.Equal(t, "", allNewAccounts[2].Name)
assert.Equal(t, "CNY", allNewAccounts[2].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[3].Uid)
assert.Equal(t, "Alipay", allNewAccounts[3].Name)
assert.Equal(t, "CNY", allNewAccounts[3].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[4].Uid)
assert.Equal(t, "Test Account3", allNewAccounts[4].Name)
assert.Equal(t, "CNY", allNewAccounts[4].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[5].Uid)
assert.Equal(t, "Test Account4", allNewAccounts[5].Name)
assert.Equal(t, "CNY", allNewAccounts[5].Currency)
}
func TestAlipayCsvFileImporterParseImportedData_ParseDescription(t *testing.T) {
@@ -18,10 +18,15 @@ const alipayTransactionDataStatusClosedName = "交易关闭"
const alipayTransactionDataStatusRefundSuccessName = "退款成功"
const alipayTransactionDataStatusTaxRefundSuccessName = "退税成功"
const alipayTransactionDataProductNameEarningText = "-收益发放"
const alipayTransactionDataProductNamePurchaseInvestmentText = "-买入"
const alipayTransactionDataProductNamePurchaseInvestmentRefundText = "-买入退款"
const alipayTransactionDataProductNameSellInvestmentRefundText = "-卖出"
const alipayTransactionDataProductNameTransferToAlipayPrefix = "充值-"
const alipayTransactionDataProductNameTransferFromAlipayPrefix = "提现-"
const alipayTransactionDataProductNameTransferInText = "转入"
const alipayTransactionDataProductNameTransferOutText = "转出"
const alipayTransactionDataProductNameTransferText = "转账"
const alipayTransactionDataProductNameRepaymentText = "还款"
// alipayTransactionDataRowParser defines the structure of alipay transaction data row parser
@@ -127,11 +132,29 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
}
if statusName == alipayTransactionDataStatusRefundSuccessName {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = ""
if len(productName) > len(alipayTransactionDataProductNamePurchaseInvestmentText) && strings.Index(productName, alipayTransactionDataProductNamePurchaseInvestmentText) == len(productName)-len(alipayTransactionDataProductNamePurchaseInvestmentText) { // purchase investment
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
} else if len(productName) > len(alipayTransactionDataProductNamePurchaseInvestmentRefundText) && strings.Index(productName, alipayTransactionDataProductNamePurchaseInvestmentRefundText) == len(productName)-len(alipayTransactionDataProductNamePurchaseInvestmentRefundText) { // purchase investment refund
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = targetName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = relatedAccountName
} else {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = ""
}
} else {
if strings.Index(productName, alipayTransactionDataProductNameTransferToAlipayPrefix) == 0 { // transfer to alipay wallet
if len(productName) > len(alipayTransactionDataProductNameEarningText) && strings.Index(productName, alipayTransactionDataProductNameEarningText) == len(productName)-len(alipayTransactionDataProductNameEarningText) { // earning
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
} else if len(productName) > len(alipayTransactionDataProductNamePurchaseInvestmentText) && strings.Index(productName, alipayTransactionDataProductNamePurchaseInvestmentText) == len(productName)-len(alipayTransactionDataProductNamePurchaseInvestmentText) { // purchase investment
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
} else if strings.Index(productName, alipayTransactionDataProductNameSellInvestmentRefundText) >= 0 { // sell investment
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = targetName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = relatedAccountName
} else if strings.Index(productName, alipayTransactionDataProductNameTransferToAlipayPrefix) == 0 { // transfer to alipay wallet
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = ""
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = localeTextItems.DataConverterTextItems.Alipay
} else if strings.Index(productName, alipayTransactionDataProductNameTransferFromAlipayPrefix) == 0 { // transfer from alipay wallet
@@ -143,6 +166,9 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
} else if strings.Index(productName, alipayTransactionDataProductNameTransferOutText) >= 0 { // transfer out
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
} else if strings.Index(productName, alipayTransactionDataProductNameTransferText) >= 0 { // transfer
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
} else if strings.Index(productName, alipayTransactionDataProductNameRepaymentText) >= 0 { // repayment
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = relatedAccountName
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = targetName
@@ -1,15 +1,20 @@
package beancount
import (
"fmt"
"strconv"
"math/big"
"strings"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const maxAllowedDecimalCount = 6
const normalizeFactor = int64(1000000)
const normalizedDecimalsMaxZeroString = "000000"
const normalizedNumberToAmountFactor = int64(10000) // 1000000 / 100
var operatorPriority = map[rune]int{
'+': 1,
'-': 1,
@@ -17,6 +22,44 @@ var operatorPriority = map[rune]int{
'/': 2,
}
func normalizeNumber(textualNumber string) (*big.Int, error) {
decimalSeparatorPos := strings.Index(textualNumber, ".")
if decimalSeparatorPos < 0 {
result := big.NewInt(0)
_, ok := result.SetString(textualNumber+normalizedDecimalsMaxZeroString, 10)
if !ok {
return nil, errs.ErrAmountInvalid
}
return result, nil
}
integer := utils.SubString(textualNumber, 0, decimalSeparatorPos)
decimals := utils.SubString(textualNumber, decimalSeparatorPos+1, len(textualNumber))
if len(decimals) > maxAllowedDecimalCount {
return nil, errs.ErrAmountInvalid
}
paddedDecimals := utils.SubString(decimals+normalizedDecimalsMaxZeroString, 0, maxAllowedDecimalCount)
result := big.NewInt(0)
_, ok := result.SetString(integer+paddedDecimals, 10)
if !ok {
return nil, errs.ErrAmountInvalid
}
return result, nil
}
func denormalizeNumberToTextualAmount(num *big.Int) string {
result := big.NewInt(0).Add(num, big.NewInt(0)) // make a copy of num
result = result.Div(result, big.NewInt(normalizedNumberToAmountFactor))
return utils.FormatAmount(result.Int64())
}
func toPostfixExprTokens(ctx core.Context, expr string) ([]string, error) {
finalTokens := make([]string, 0)
operatorStack := make([]rune, 0)
@@ -117,8 +160,8 @@ func toPostfixExprTokens(ctx core.Context, expr string) ([]string, error) {
return finalTokens, nil
}
func evaluatePostfixExpr(ctx core.Context, tokens []string) (float64, error) {
stack := make([]float64, 0)
func evaluatePostfixExpr(ctx core.Context, tokens []string) (*big.Int, error) {
stack := make([]*big.Int, 0)
for i := 0; i < len(tokens); i++ {
token := tokens[i]
@@ -127,7 +170,7 @@ func evaluatePostfixExpr(ctx core.Context, tokens []string) (float64, error) {
case "+", "-", "*", "/": // operators
if len(stack) < 2 {
log.Warnf(ctx, "[beancount_amount_expression_evaluator.evaluatePostfixExpr] cannot evaluate expression \"%s\", because not enough operands", strings.Join(tokens, " "))
return 0, errs.ErrInvalidAmountExpression
return nil, errs.ErrInvalidAmountExpression
}
// pop the top two operands
@@ -138,39 +181,41 @@ func evaluatePostfixExpr(ctx core.Context, tokens []string) (float64, error) {
stack = stack[:len(stack)-1]
// evaluate the operation
var result float64
result := big.NewInt(0)
switch token {
case "+":
result = a + b
result.Add(a, b)
case "-":
result = a - b
result.Sub(a, b)
case "*":
result = a * b
result.Mul(a, b)
result.Div(result, big.NewInt(normalizeFactor))
case "/":
if b == 0 {
if b.Int64() == 0 {
log.Warnf(ctx, "[beancount_amount_expression_evaluator.evaluatePostfixExpr] cannot evaluate expression \"%s\", because division by zero", strings.Join(tokens, " "))
return 0, errs.ErrInvalidAmountExpression
return nil, errs.ErrInvalidAmountExpression
}
result = a / b
result.Mul(a, big.NewInt(normalizeFactor))
result.Div(result, b)
}
// push the result back to the stack
stack = append(stack, result)
default: // operands
num, err := strconv.ParseFloat(token, 64)
normalizedNum, err := normalizeNumber(token)
if err != nil {
log.Warnf(ctx, "[beancount_amount_expression_evaluator.evaluatePostfixExpr] cannot evaluate expression \"%s\", because containing invalid number", strings.Join(tokens, " "))
return 0, errs.ErrInvalidAmountExpression
return nil, errs.ErrInvalidAmountExpression
}
stack = append(stack, num)
stack = append(stack, normalizedNum)
}
}
if len(stack) != 1 {
log.Warnf(ctx, "[beancount_amount_expression_evaluator.evaluatePostfixExpr] cannot evaluate expression \"%s\", because missing operator", strings.Join(tokens, " "))
return 0, errs.ErrInvalidAmountExpression
return nil, errs.ErrInvalidAmountExpression
}
return stack[0], nil
@@ -193,5 +238,5 @@ func evaluateBeancountAmountExpression(ctx core.Context, expr string) (string, e
return "", err
}
return fmt.Sprintf("%.2f", result), nil
return denormalizeNumberToTextualAmount(result), nil
}
@@ -1,6 +1,7 @@
package beancount
import (
"math/big"
"testing"
"github.com/stretchr/testify/assert"
@@ -97,23 +98,23 @@ func TestEvaluatePostfixExpr_ValidExpression(t *testing.T) {
result, err := evaluatePostfixExpr(context, []string{"1", "2", "+"})
assert.Nil(t, err)
assert.Equal(t, float64(3), result)
assert.Equal(t, big.NewInt(3000000), result)
result, err = evaluatePostfixExpr(context, []string{"5", "3", "-"})
assert.Nil(t, err)
assert.Equal(t, float64(2), result)
assert.Equal(t, big.NewInt(2000000), result)
result, err = evaluatePostfixExpr(context, []string{"4", "3", "*"})
assert.Nil(t, err)
assert.Equal(t, float64(12), result)
assert.Equal(t, big.NewInt(12000000), result)
result, err = evaluatePostfixExpr(context, []string{"6", "2", "/"})
assert.Nil(t, err)
assert.Equal(t, float64(3), result)
assert.Equal(t, big.NewInt(3000000), result)
result, err = evaluatePostfixExpr(context, []string{"1", "2", "3", "*", "+", "4", "2", "/", "-"})
assert.Nil(t, err)
assert.Equal(t, float64(5), result)
assert.Equal(t, big.NewInt(5000000), result)
}
func TestEvaluatePostfixExpr_InvalidExpression(t *testing.T) {
@@ -179,6 +180,18 @@ func TestEvaluateBeancountAmountExpression_ValidExpression(t *testing.T) {
result, err = evaluateBeancountAmountExpression(context, "(((2+3)))*(((((-5+7)))))")
assert.Nil(t, err)
assert.Equal(t, "10.00", result)
result, err = evaluateBeancountAmountExpression(context, "3.5+0.1")
assert.Nil(t, err)
assert.Equal(t, "3.60", result)
result, err = evaluateBeancountAmountExpression(context, "3.55+0.11")
assert.Nil(t, err)
assert.Equal(t, "3.66", result)
result, err = evaluateBeancountAmountExpression(context, "3.555+0.111")
assert.Nil(t, err)
assert.Equal(t, "3.66", result)
}
func TestEvaluateBeancountAmountExpression_InvalidExpression(t *testing.T) {
@@ -213,4 +226,10 @@ func TestEvaluateBeancountAmountExpression_InvalidExpression(t *testing.T) {
_, err = evaluateBeancountAmountExpression(context, "1)*(2")
assert.Equal(t, errs.ErrInvalidAmountExpression, err)
_, err = evaluateBeancountAmountExpression(context, "0.abcd+1")
assert.Equal(t, errs.ErrInvalidAmountExpression, err)
_, err = evaluateBeancountAmountExpression(context, "0.1234567+1")
assert.Equal(t, errs.ErrInvalidAmountExpression, err)
}
@@ -0,0 +1,96 @@
package _default
import (
"encoding/json"
"time"
"github.com/mayswind/ezbookkeeping/pkg/converters/converter"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
var allJsonDataSupportedColumns = []datatable.TransactionDataTableColumn{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE,
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME,
datatable.TRANSACTION_DATA_TABLE_AMOUNT,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME,
datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT,
datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION,
datatable.TRANSACTION_DATA_TABLE_TAGS,
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION,
}
// defaultTransactionDataJsonImporter defines the structure of ezbookkeeping default json importer for transaction data
type defaultTransactionDataJsonImporter struct{}
// Initialize an ezbookkeeping default transaction data json file importer singleton instance
var (
DefaultTransactionDataJsonFileImporter = &defaultTransactionDataJsonImporter{}
)
// ParseImportedData returns the imported data by parsing the transaction json data
func (c *defaultTransactionDataJsonImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
var importRequest models.ImportTransactionRequest
if err := json.Unmarshal(data, &importRequest); err != nil {
return nil, nil, nil, nil, nil, nil, errs.ErrInvalidJSONFile
}
transactionDataTable, err := c.createNewDefaultTransactionDataTable(importRequest)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(
ezbookkeepingTransactionTypeNameMapping,
ezbookkeepingGeoLocationSeparator,
ezbookkeepingGeoLocationOrder,
ezbookkeepingTagSeparator,
)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
func (c *defaultTransactionDataJsonImporter) createNewDefaultTransactionDataTable(importRequest models.ImportTransactionRequest) (datatable.TransactionDataTable, error) {
transactionDataTable := datatable.CreateNewWritableTransactionDataTable(allJsonDataSupportedColumns)
if importRequest.Transactions == nil || len(importRequest.Transactions) < 1 {
return nil, errs.ErrNotFoundTransactionDataInFile
}
for i := 0; i < len(importRequest.Transactions); i++ {
transaction := importRequest.Transactions[i]
utcOffset, err := utils.StringToInt(transaction.UtcOffset)
if err != nil {
return nil, errs.ErrTransactionTimeZoneInvalid
}
timezone := time.FixedZone("Transaction Timezone", utcOffset*60)
row := make(map[datatable.TransactionDataTableColumn]string, len(allJsonDataSupportedColumns))
row[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = transaction.Time
row[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE] = utils.FormatTimezoneOffset(timezone)
row[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = transaction.Type
row[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = transaction.CategoryName
row[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = transaction.SourceAccountName
row[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = transaction.SourceAmount
row[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = transaction.DestinationAccountName
row[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = transaction.DestinationAmount
row[datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION] = transaction.GeoLocation
row[datatable.TRANSACTION_DATA_TABLE_TAGS] = transaction.TagNames
row[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = transaction.Comment
transactionDataTable.Add(row)
}
return transactionDataTable, nil
}
@@ -0,0 +1,64 @@
package jdcom
import (
"bytes"
"golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"
"github.com/mayswind/ezbookkeeping/pkg/converters/converter"
"github.com/mayswind/ezbookkeeping/pkg/converters/csv"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
// jdComFinanceTransactionDataCsvFileImporter defines the structure of jd.com finance csv importer for transaction data
type jdComFinanceTransactionDataCsvFileImporter struct {
fileHeaderLineBeginning string
dataHeaderStartContentBeginning string
}
// Initialize a jd.com finance transaction data csv file importer singleton instance
var (
JDComFinanceTransactionDataCsvFileImporter = &jdComFinanceTransactionDataCsvFileImporter{}
)
// ParseImportedData returns the imported data by parsing the jd.com finance transaction csv data
func (c *jdComFinanceTransactionDataCsvFileImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
fallback := unicode.UTF8.NewDecoder()
reader := transform.NewReader(bytes.NewReader(data), unicode.BOMOverride(fallback))
csvDataTable, err := csv.CreateNewCsvBasicDataTable(ctx, reader, false)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
dataTable, err := createNewJDComFinanceTransactionBasicDataTable(ctx, csvDataTable)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
commonDataTable := datatable.CreateNewCommonDataTableFromBasicDataTable(dataTable)
if !commonDataTable.HasColumn(jdComFinanceTransactionTimeColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionMerchantNameColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionMemoColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionAmountColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionRelatedAccountColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionStatusColumnName) ||
!commonDataTable.HasColumn(jdComFinanceTransactionTypeColumnName) {
log.Errorf(ctx, "[jdcom_finance_transaction_data_csv_file_importer.ParseImportedData] cannot parse jd.com finance csv data, because missing essential columns in header row")
return nil, nil, nil, nil, nil, nil, errs.ErrMissingRequiredFieldInHeaderRow
}
transactionRowParser := createJDComFinanceTransactionDataRowParser(dataTable.HeaderColumnNames())
transactionDataTable := datatable.CreateNewTransactionDataTableFromCommonDataTable(commonDataTable, jdComFinanceTransactionSupportedColumns, transactionRowParser)
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(jdComFinanceTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -0,0 +1,508 @@
package jdcom
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
func TestJDComFinanceCsvFileImporterParseImportedData_MinimumValidData(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,余额,交易成功,收入,其他\n" +
"2025-09-01 12:34:56,xxx,xxx,123.45,银行卡,交易成功,支出,其他网购\n" +
"2025-09-01 23:59:59,xxx,京东钱包余额充值,0.05,银行卡,交易成功,不计收支,余额\n" +
"2025-09-02 23:59:59,xxx,京东余额提现,0.03,银行卡,交易成功,不计收支,余额\n"
allNewTransactions, allNewAccounts, allNewSubExpenseCategories, allNewSubIncomeCategories, allNewSubTransferCategories, allNewTags, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 4, len(allNewTransactions))
assert.Equal(t, 3, len(allNewAccounts))
assert.Equal(t, 1, len(allNewSubExpenseCategories))
assert.Equal(t, 1, len(allNewSubIncomeCategories))
assert.Equal(t, 1, len(allNewSubTransferCategories))
assert.Equal(t, 0, len(allNewTags))
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, "2025-09-01 01:23:45", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime), time.UTC))
assert.Equal(t, int64(12), allNewTransactions[0].Amount)
assert.Equal(t, "余额", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "其他", allNewTransactions[0].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[1].Type)
assert.Equal(t, "2025-09-01 12:34:56", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime), time.UTC))
assert.Equal(t, int64(12345), allNewTransactions[1].Amount)
assert.Equal(t, "银行卡", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "其他网购", allNewTransactions[1].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[2].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[2].Type)
assert.Equal(t, "2025-09-01 23:59:59", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[2].TransactionTime), time.UTC))
assert.Equal(t, int64(5), allNewTransactions[2].Amount)
assert.Equal(t, "银行卡", allNewTransactions[2].OriginalSourceAccountName)
assert.Equal(t, "xxx", allNewTransactions[2].OriginalDestinationAccountName)
assert.Equal(t, "余额", allNewTransactions[2].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[3].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[3].Type)
assert.Equal(t, "2025-09-02 23:59:59", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[3].TransactionTime), time.UTC))
assert.Equal(t, int64(3), allNewTransactions[3].Amount)
assert.Equal(t, "xxx", allNewTransactions[3].OriginalSourceAccountName)
assert.Equal(t, "银行卡", allNewTransactions[3].OriginalDestinationAccountName)
assert.Equal(t, "余额", allNewTransactions[3].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewAccounts[0].Uid)
assert.Equal(t, "余额", allNewAccounts[0].Name)
assert.Equal(t, "CNY", allNewAccounts[0].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[1].Uid)
assert.Equal(t, "银行卡", allNewAccounts[1].Name)
assert.Equal(t, "CNY", allNewAccounts[1].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[2].Uid)
assert.Equal(t, "xxx", allNewAccounts[2].Name)
assert.Equal(t, "CNY", allNewAccounts[2].Currency)
assert.Equal(t, int64(1234567890), allNewSubExpenseCategories[0].Uid)
assert.Equal(t, "其他网购", allNewSubExpenseCategories[0].Name)
assert.Equal(t, int64(1234567890), allNewSubIncomeCategories[0].Uid)
assert.Equal(t, "其他", allNewSubIncomeCategories[0].Name)
assert.Equal(t, int64(1234567890), allNewSubTransferCategories[0].Uid)
assert.Equal(t, "余额", allNewSubTransferCategories[0].Name)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseRefundTransaction(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data1 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,退款成功,不计收支\n" +
"2025-09-01 02:34:56,xxx,xxx,0.12(已全额退款),银行卡,交易成功,不计收支\n" +
"2025-09-02 01:23:45,xxx,xxx,3.45,银行卡,退款成功,不计收支\n" +
"2025-09-02 02:34:56,xxx,xxx,123.45(已退款3.45),银行卡,交易成功,支出\n"
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[0].Type)
assert.Equal(t, "2025-09-01 01:23:45", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime), time.UTC))
assert.Equal(t, int64(-12), allNewTransactions[0].Amount)
assert.Equal(t, "银行卡", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[1].Type)
assert.Equal(t, "2025-09-01 02:34:56", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime), time.UTC))
assert.Equal(t, int64(12), allNewTransactions[1].Amount)
assert.Equal(t, "银行卡", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, int64(1234567890), allNewTransactions[2].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[2].Type)
assert.Equal(t, "2025-09-02 01:23:45", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[2].TransactionTime), time.UTC))
assert.Equal(t, int64(-345), allNewTransactions[2].Amount)
assert.Equal(t, "银行卡", allNewTransactions[2].OriginalSourceAccountName)
assert.Equal(t, int64(1234567890), allNewTransactions[3].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[3].Type)
assert.Equal(t, "2025-09-02 02:34:56", utils.FormatUnixTimeToLongDateTime(utils.GetUnixTimeFromTransactionTime(allNewTransactions[3].TransactionTime), time.UTC))
assert.Equal(t, int64(12345), allNewTransactions[3].Amount)
assert.Equal(t, "银行卡", allNewTransactions[3].OriginalSourceAccountName)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseInvalidTime(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data1 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01T01:23:45,xxx,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
data2 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"09/01/2025 01:23:45,xxx,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data2), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseInvalidType(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,交易成功,转账\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseInvalidAmount(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,¥0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseAccountName(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
// transfer to jd.com finance wallet
data1 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,京东钱包余额充值,0.05,银行卡,交易成功,不计收支,余额\n"
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "银行卡", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "xxx", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
// transfer from jd.com finance wallet
data2 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,京东余额提现,0.05,银行卡,交易成功,不计收支,余额\n"
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data2), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "xxx", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "银行卡", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
// transfer from other account
data3 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,京东小金库-转入,0.05,余额,交易成功,不计收支,小金库\n"
assert.Nil(t, err)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data3), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "余额", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "xxx", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
// transfer to other account
data4 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,京东小金库-转出,0.05,余额,交易成功,不计收支,小金库\n"
assert.Nil(t, err)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data4), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "xxx", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "余额", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
// refund
data5 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,价保退款,0.05,银行卡,交易成功,不计收支,其他\n"
assert.Nil(t, err)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data5), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "银行卡", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[0].Type)
// repayment
data6 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支,交易分类\n" +
"2025-09-01 01:23:45,xxx,白条主动还款,0.05,银行卡,交易成功,不计收支,白条\n"
assert.Nil(t, err)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data6), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "银行卡", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "xxx", allNewTransactions[0].OriginalDestinationAccountName)
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
}
func TestJDComFinanceCsvFileImporterParseImportedData_ParseDescription(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data1 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,,0.12,银行卡,交易成功,支出\n"
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "", allNewTransactions[0].Comment)
data2 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,交易说明,金额,收/付款方式,交易状态,收/支,备注\n" +
"2025-09-01 01:23:45,xxx,xxx,Test,0.12,银行卡,交易成功,支出,\"foo\"\"bar,\ntest\"\n"
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data2), 0, nil, nil, nil, nil, nil)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "foo\"bar,\ntest", allNewTransactions[0].Comment)
data3 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,交易说明,金额,收/付款方式,交易状态,收/支,备注\n" +
"2025-09-01 01:23:45,xxx,xxx,Test,0.12,银行卡,交易成功,支出,\n"
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data3), 0, nil, nil, nil, nil, nil)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "Test", allNewTransactions[0].Comment)
}
func TestJDComFinanceCsvFileImporterParseImportedData_SkipUnknownStatusTransaction(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,xxxx,支出\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_SkipUnknownMemoTransferTransaction(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,交易成功,不计收支\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_MissingFileHeader(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1,
DefaultCurrency: "CNY",
}
data := "交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrInvalidFileHeader.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrInvalidFileHeader.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_MissingRequiredColumn(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1,
DefaultCurrency: "CNY",
}
// Missing Time Column
data1 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"xxx,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data1), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrInvalidFileHeader.Message)
// Missing Merchant Name Column
data2 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,交易说明,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data2), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Transaction Memo Column
data3 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,金额,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,0.12,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data3), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Amount Column
data4 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,收/付款方式,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,银行卡,交易成功,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data4), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Related Account Column
data5 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,交易状态,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,交易成功,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data5), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Status Column
data6 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,收/支\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,支出\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data6), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Type Column
data7 := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态\n" +
"2025-09-01 01:23:45,xxx,xxx,0.12,银行卡,交易成功\n"
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(data7), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
}
func TestJDComFinanceCsvFileImporterParseImportedData_NoTransactionData(t *testing.T) {
converter := JDComFinanceTransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
data := "导出信息:\n" +
"京东账号名:xxxxxx\n" +
"日期区间:2025-01-01 至 2025-09-01\n" +
"\n" +
"交易时间,商户名称,交易说明,金额,收/付款方式,交易状态,收/支\n"
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(data), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
}
@@ -0,0 +1,74 @@
package jdcom
import (
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/csv"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
)
func createNewJDComFinanceTransactionBasicDataTable(ctx core.Context, originalDataTable datatable.BasicDataTable) (datatable.BasicDataTable, error) {
iterator := originalDataTable.DataRowIterator()
allOriginalLines := make([][]string, 0)
hasFileHeader := false
foundDataHeaderLine := false
for iterator.HasNext() {
row := iterator.Next()
if !hasFileHeader {
if row.ColumnCount() <= 0 {
continue
} else if strings.Index(row.GetData(0), jdComFinanceTransactionDataCsvFileHeader) == 0 {
hasFileHeader = true
continue
} else {
log.Warnf(ctx, "[jdcom_finance_transaction_data_extrator.createNewJDComFinanceTransactionBasicDataTable] read unexpected line in row \"%s\" before read file header", iterator.CurrentRowId())
continue
}
}
if !foundDataHeaderLine {
if row.ColumnCount() <= 0 {
continue
} else if row.GetData(0) == jdComFinanceTransactionTimeColumnName {
foundDataHeaderLine = true
} else {
continue
}
}
if foundDataHeaderLine {
if row.ColumnCount() <= 0 {
continue
}
items := make([]string, row.ColumnCount())
for i := 0; i < row.ColumnCount(); i++ {
items[i] = strings.TrimRight(strings.Trim(row.GetData(i), " "), "\t")
}
if len(allOriginalLines) > 0 && len(items) < len(allOriginalLines[0]) {
log.Errorf(ctx, "[jdcom_finance_transaction_data_extrator.createNewJDComFinanceTransactionBasicDataTable] cannot parse row \"%s\", because may missing some columns (column count %d in data row is less than header column count %d)", iterator.CurrentRowId(), len(items), len(allOriginalLines[0]))
return nil, errs.ErrFewerFieldsInDataRowThanInHeaderRow
}
allOriginalLines = append(allOriginalLines, items)
}
}
if !hasFileHeader || !foundDataHeaderLine {
return nil, errs.ErrInvalidFileHeader
}
if len(allOriginalLines) < 2 {
log.Errorf(ctx, "[jdcom_finance_transaction_data_extrator.createNewJDComFinanceTransactionBasicDataTable] cannot parse import data, because data table row count is less 1")
return nil, errs.ErrNotFoundTransactionDataInFile
}
return csv.CreateNewCustomCsvBasicDataTable(allOriginalLines, true), nil
}
@@ -0,0 +1,148 @@
package jdcom
import (
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const jdComFinanceTransactionDataCsvFileHeader = "导出信息:"
const jdComFinanceTransactionTimeColumnName = "交易时间"
const jdComFinanceTransactionMerchantNameColumnName = "商户名称"
const jdComFinanceTransactionMemoColumnName = "交易说明"
const jdComFinanceTransactionAmountColumnName = "金额"
const jdComFinanceTransactionRelatedAccountColumnName = "收/付款方式"
const jdComFinanceTransactionStatusColumnName = "交易状态"
const jdComFinanceTransactionTypeColumnName = "收/支"
const jdComFinanceTransactionCategoryColumnName = "交易分类"
const jdComFinanceTransactionDescriptionColumnName = "备注"
const jdComFinanceTransactionAmountRefundAll = "(已全额退款)"
const jdComFinanceTransactionMemoTransferToWalletPrefix = "充值"
const jdComFinanceTransactionMemoTransferFromWalletPrefix = "提现"
const jdComFinanceTransactionMemoTransferInText = "转入"
const jdComFinanceTransactionMemoTransferOutText = "转出"
const jdComFinanceTransactionMemoRepaymentText = "还款"
const jdComFinanceTransactionMemoRefundText = "退款"
const jdComFinanceTransactionDataStatusSuccessName = "交易成功"
const jdComFinanceTransactionDataStatusRefundSuccessName = "退款成功"
var jdComFinanceTransactionSupportedColumns = map[datatable.TransactionDataTableColumn]bool{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: true,
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION: true,
}
var jdComFinanceTransactionTypeNameMapping = map[models.TransactionType]string{
models.TRANSACTION_TYPE_INCOME: "收入",
models.TRANSACTION_TYPE_EXPENSE: "支出",
models.TRANSACTION_TYPE_TRANSFER: "不计收支",
}
// jdComFinanceTransactionDataRowParser defines the structure of jd.com finance transaction data row parser
type jdComFinanceTransactionDataRowParser struct {
existedOriginalDataColumns map[string]bool
}
// Parse returns the converted transaction data row
func (p *jdComFinanceTransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataRow datatable.CommonDataTableRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
if dataRow.GetData(jdComFinanceTransactionTypeColumnName) != jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] &&
dataRow.GetData(jdComFinanceTransactionTypeColumnName) != jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE] &&
dataRow.GetData(jdComFinanceTransactionTypeColumnName) != jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER] {
log.Warnf(ctx, "[jdcom_finance_transaction_data_row_parser.Parse] skip parsing transaction in row \"%s\", because type is \"%s\"", rowId, dataRow.GetData(jdComFinanceTransactionTypeColumnName))
return nil, false, nil
}
statusName := dataRow.GetData(jdComFinanceTransactionStatusColumnName)
if statusName != jdComFinanceTransactionDataStatusSuccessName &&
statusName != jdComFinanceTransactionDataStatusRefundSuccessName {
log.Warnf(ctx, "[jdcom_finance_transaction_data_row_parser.Parse] skip parsing transaction in row \"%s\", because status is \"%s\"", rowId, statusName)
return nil, false, nil
}
data := make(map[datatable.TransactionDataTableColumn]string, len(jdComFinanceTransactionSupportedColumns))
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = dataRow.GetData(jdComFinanceTransactionTimeColumnName)
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = dataRow.GetData(jdComFinanceTransactionTypeColumnName)
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(jdComFinanceTransactionCategoryColumnName)
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionRelatedAccountColumnName)
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = ""
if strings.Index(dataRow.GetData(jdComFinanceTransactionAmountColumnName), "(") >= 0 {
// If a transaction includes a refund, the original transaction amount will like "-xx.xx(已全额退款)" or "-xx.xx(已退款yy.yy)", along with another refund transaction
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = strings.Split(dataRow.GetData(jdComFinanceTransactionAmountColumnName), "(")[0]
} else {
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = dataRow.GetData(jdComFinanceTransactionAmountColumnName)
}
if p.hasOriginalColumn(jdComFinanceTransactionDescriptionColumnName) && dataRow.GetData(jdComFinanceTransactionDescriptionColumnName) != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(jdComFinanceTransactionDescriptionColumnName)
} else if p.hasOriginalColumn(jdComFinanceTransactionMemoColumnName) && dataRow.GetData(jdComFinanceTransactionMemoColumnName) != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(jdComFinanceTransactionMemoColumnName)
} else {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = ""
}
if data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] == jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER] {
memo := dataRow.GetData(jdComFinanceTransactionMemoColumnName)
if statusName == jdComFinanceTransactionDataStatusRefundSuccessName || strings.Index(memo, jdComFinanceTransactionMemoRefundText) >= 0 { // refund
amount, err := utils.ParseAmount(data[datatable.TRANSACTION_DATA_TABLE_AMOUNT])
if err == nil {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE]
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-amount)
}
} else if strings.Index(dataRow.GetData(jdComFinanceTransactionAmountColumnName), jdComFinanceTransactionAmountRefundAll) > 0 { // expense transaction (but include a full refund)
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = jdComFinanceTransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE]
} else { // transfer
if strings.Index(memo, jdComFinanceTransactionMemoTransferToWalletPrefix) >= 0 { // transfer to jd.com finance wallet
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionMerchantNameColumnName)
} else if strings.Index(memo, jdComFinanceTransactionMemoTransferFromWalletPrefix) >= 0 { // transfer from jd.com finance wallet
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionMerchantNameColumnName)
} else if strings.Index(memo, jdComFinanceTransactionMemoTransferInText) >= 0 { // transfer in
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionMerchantNameColumnName)
} else if strings.Index(memo, jdComFinanceTransactionMemoTransferOutText) >= 0 { // transfer out
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionMerchantNameColumnName)
} else if strings.Index(memo, jdComFinanceTransactionMemoRepaymentText) >= 0 { // repayment
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = dataRow.GetData(jdComFinanceTransactionMerchantNameColumnName)
} else {
log.Warnf(ctx, "[jdcom_finance_transaction_data_row_parser.Parse] skip parsing transaction in row \"%s\", because memo (\"%s\") of this transfer transaction is unknown", rowId, memo)
return nil, false, nil
}
}
}
return data, true, nil
}
func (p *jdComFinanceTransactionDataRowParser) hasOriginalColumn(columnName string) bool {
_, exists := p.existedOriginalDataColumns[columnName]
return exists
}
// createJDComFinanceTransactionDataRowParser returns jd.com finance transaction data row parser
func createJDComFinanceTransactionDataRowParser(headerColumnNames []string) datatable.CommonTransactionDataRowParser {
existedOriginalDataColumns := make(map[string]bool, len(headerColumnNames))
for i := 0; i < len(headerColumnNames); i++ {
existedOriginalDataColumns[headerColumnNames[i]] = true
}
return &jdComFinanceTransactionDataRowParser{
existedOriginalDataColumns: existedOriginalDataColumns,
}
}
+13 -4
View File
@@ -4,6 +4,7 @@ import (
"bufio"
"bytes"
"encoding/xml"
"io"
"regexp"
"strings"
@@ -269,19 +270,27 @@ func readOFX1FileHeader(ctx core.Context, data []byte) (fileHeader *ofxFileHeade
func readOFX2FileHeader(ctx core.Context, data []byte) (fileHeader *ofxFileHeader, err error) {
reader := bytes.NewReader(data)
scanner := bufio.NewScanner(reader)
bufReader := bufio.NewReader(reader)
fileHeader = &ofxFileHeader{}
headerLine := ""
for scanner.Scan() {
line := scanner.Text()
for {
line, err := bufReader.ReadString('\n')
ofxHeaderStartIndex := strings.Index(line, "<?OFX ")
if ofxHeaderStartIndex >= 0 {
headerLine = ofx2HeaderPattern.FindString(line)
break
}
if err != nil {
if err == io.EOF {
break
} else {
log.Errorf(ctx, "[ofx_data_reader.readOFX2FileHeader] cannot read ofx 2.x file, because %s", err.Error())
return nil, errs.ErrInvalidOFXFile
}
}
}
if headerLine == "" {
@@ -12,6 +12,7 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/converters/fireflyIII"
"github.com/mayswind/ezbookkeeping/pkg/converters/gnucash"
"github.com/mayswind/ezbookkeeping/pkg/converters/iif"
"github.com/mayswind/ezbookkeeping/pkg/converters/jdcom"
"github.com/mayswind/ezbookkeeping/pkg/converters/mt"
"github.com/mayswind/ezbookkeeping/pkg/converters/ofx"
"github.com/mayswind/ezbookkeeping/pkg/converters/qif"
@@ -37,6 +38,8 @@ func GetTransactionDataImporter(fileType string) (converter.TransactionDataImpor
return _default.DefaultTransactionDataCSVFileConverter, nil
} else if fileType == "ezbookkeeping_tsv" {
return _default.DefaultTransactionDataTSVFileConverter, nil
} else if fileType == "ezbookkeeping_json" {
return _default.DefaultTransactionDataJsonFileImporter, nil
} else if fileType == "ofx" {
return ofx.OFXTransactionDataImporter, nil
} else if fileType == "qfx" {
@@ -73,6 +76,8 @@ func GetTransactionDataImporter(fileType string) (converter.TransactionDataImpor
return wechat.WeChatPayTransactionDataXlsxFileImporter, nil
} else if fileType == "wechat_pay_app_csv" {
return wechat.WeChatPayTransactionDataCsvFileImporter, nil
} else if fileType == "jdcom_finance_app_csv" {
return jdcom.JDComFinanceTransactionDataCsvFileImporter, nil
} else {
return nil, errs.ErrImportFileTypeNotSupported
}
+13 -4
View File
@@ -7,10 +7,12 @@ type CalendarDisplayType byte
// Calendar Display Type
const (
CALENDAR_DISPLAY_TYPE_DEFAULT CalendarDisplayType = 0
CALENDAR_DISPLAY_TYPE_GREGORAIN CalendarDisplayType = 1
CALENDAR_DISPLAY_TYPE_BUDDHIST CalendarDisplayType = 2
CALENDAR_DISPLAY_TYPE_INVALID CalendarDisplayType = 255
CALENDAR_DISPLAY_TYPE_DEFAULT CalendarDisplayType = 0
CALENDAR_DISPLAY_TYPE_GREGORAIN CalendarDisplayType = 1
CALENDAR_DISPLAY_TYPE_BUDDHIST CalendarDisplayType = 2
CALENDAR_DISPLAY_TYPE_GREGORAIN_WITH_CHINESE CalendarDisplayType = 3
CALENDAR_DISPLAY_TYPE_GREGORAIN_WITH_PERSIAN CalendarDisplayType = 4
CALENDAR_DISPLAY_TYPE_INVALID CalendarDisplayType = 255
)
// String returns a textual representation of the calendar display type enum
@@ -22,6 +24,10 @@ func (f CalendarDisplayType) String() string {
return "Gregorian"
case CALENDAR_DISPLAY_TYPE_BUDDHIST:
return "Buddhist"
case CALENDAR_DISPLAY_TYPE_GREGORAIN_WITH_CHINESE:
return "Gregorian with Chinese Calendar"
case CALENDAR_DISPLAY_TYPE_GREGORAIN_WITH_PERSIAN:
return "Gregorian with Persian Calendar"
case CALENDAR_DISPLAY_TYPE_INVALID:
return "Invalid"
default:
@@ -37,6 +43,7 @@ const (
DATE_DISPLAY_TYPE_DEFAULT DateDisplayType = 0
DATE_DISPLAY_TYPE_GREGORAIN DateDisplayType = 1
DATE_DISPLAY_TYPE_BUDDHIST DateDisplayType = 2
DATE_DISPLAY_TYPE_PERSIAN DateDisplayType = 3
DATE_DISPLAY_TYPE_INVALID DateDisplayType = 255
)
@@ -49,6 +56,8 @@ func (f DateDisplayType) String() string {
return "Gregorian"
case DATE_DISPLAY_TYPE_BUDDHIST:
return "Buddhist"
case DATE_DISPLAY_TYPE_PERSIAN:
return "Persian"
case DATE_DISPLAY_TYPE_INVALID:
return "Invalid"
default:
+4
View File
@@ -0,0 +1,4 @@
package core
// O is a shortcut for map[string]any
type O map[string]any
+15 -14
View File
@@ -76,23 +76,24 @@ type UserFeatureRestrictionType uint64
// User Feature Restriction Type
const (
USER_FEATURE_RESTRICTION_TYPE_UPDATE_PASSWORD UserFeatureRestrictionType = 1
USER_FEATURE_RESTRICTION_TYPE_UPDATE_EMAIL UserFeatureRestrictionType = 2
USER_FEATURE_RESTRICTION_TYPE_UPDATE_PROFILE_BASIC_INFO UserFeatureRestrictionType = 3
USER_FEATURE_RESTRICTION_TYPE_UPDATE_AVATAR UserFeatureRestrictionType = 4
USER_FEATURE_RESTRICTION_TYPE_REVOKE_OTHER_SESSION UserFeatureRestrictionType = 5
USER_FEATURE_RESTRICTION_TYPE_ENABLE_2FA UserFeatureRestrictionType = 6
USER_FEATURE_RESTRICTION_TYPE_DISABLE_2FA UserFeatureRestrictionType = 7
USER_FEATURE_RESTRICTION_TYPE_FORGET_PASSWORD UserFeatureRestrictionType = 8
USER_FEATURE_RESTRICTION_TYPE_IMPORT_TRANSACTION UserFeatureRestrictionType = 9
USER_FEATURE_RESTRICTION_TYPE_EXPORT_TRANSACTION UserFeatureRestrictionType = 10
USER_FEATURE_RESTRICTION_TYPE_CLEAR_ALL_DATA UserFeatureRestrictionType = 11
USER_FEATURE_RESTRICTION_TYPE_SYNC_APPLICATION_SETTINGS UserFeatureRestrictionType = 12
USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS UserFeatureRestrictionType = 13
USER_FEATURE_RESTRICTION_TYPE_UPDATE_PASSWORD UserFeatureRestrictionType = 1
USER_FEATURE_RESTRICTION_TYPE_UPDATE_EMAIL UserFeatureRestrictionType = 2
USER_FEATURE_RESTRICTION_TYPE_UPDATE_PROFILE_BASIC_INFO UserFeatureRestrictionType = 3
USER_FEATURE_RESTRICTION_TYPE_UPDATE_AVATAR UserFeatureRestrictionType = 4
USER_FEATURE_RESTRICTION_TYPE_REVOKE_OTHER_SESSION UserFeatureRestrictionType = 5
USER_FEATURE_RESTRICTION_TYPE_ENABLE_2FA UserFeatureRestrictionType = 6
USER_FEATURE_RESTRICTION_TYPE_DISABLE_2FA UserFeatureRestrictionType = 7
USER_FEATURE_RESTRICTION_TYPE_FORGET_PASSWORD UserFeatureRestrictionType = 8
USER_FEATURE_RESTRICTION_TYPE_IMPORT_TRANSACTION UserFeatureRestrictionType = 9
USER_FEATURE_RESTRICTION_TYPE_EXPORT_TRANSACTION UserFeatureRestrictionType = 10
USER_FEATURE_RESTRICTION_TYPE_CLEAR_ALL_DATA UserFeatureRestrictionType = 11
USER_FEATURE_RESTRICTION_TYPE_SYNC_APPLICATION_SETTINGS UserFeatureRestrictionType = 12
USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS UserFeatureRestrictionType = 13
USER_FEATURE_RESTRICTION_TYPE_CREATE_TRANSACTION_FROM_AI_IMAGE_RECOGNITION UserFeatureRestrictionType = 14
)
const userFeatureRestrictionTypeMinValue UserFeatureRestrictionType = USER_FEATURE_RESTRICTION_TYPE_UPDATE_PASSWORD
const userFeatureRestrictionTypeMaxValue UserFeatureRestrictionType = USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS
const userFeatureRestrictionTypeMaxValue UserFeatureRestrictionType = USER_FEATURE_RESTRICTION_TYPE_CREATE_TRANSACTION_FROM_AI_IMAGE_RECOGNITION
// String returns a textual representation of the restriction type of user features
func (t UserFeatureRestrictionType) String() string {
+8 -8
View File
@@ -128,16 +128,16 @@ func getMysqlConnectionString(dbConfig *settings.DatabaseConfig) (string, error)
}
func getPostgresConnectionString(dbConfig *settings.DatabaseConfig) (string, error) {
host, port, err := net.SplitHostPort(dbConfig.DatabaseHost)
if err != nil {
return "", errs.ErrDatabaseHostInvalid
}
if strings.HasPrefix(dbConfig.DatabaseHost, "/") { // unix socket path
return fmt.Sprintf("postgres://%s:%s@:%s/%s?sslmode=%s&host=%s",
url.QueryEscape(dbConfig.DatabaseUser), url.QueryEscape(dbConfig.DatabasePassword), port, dbConfig.DatabaseName, dbConfig.DatabaseSSLMode, host), nil
return fmt.Sprintf("postgres:///%s?sslmode=%s&host=%s&user=%s&password=%s",
dbConfig.DatabaseName, dbConfig.DatabaseSSLMode, dbConfig.DatabaseHost, url.QueryEscape(dbConfig.DatabaseUser), url.QueryEscape(dbConfig.DatabasePassword)), nil
} else {
host, port, err := net.SplitHostPort(dbConfig.DatabaseHost)
if err != nil {
return "", errs.ErrDatabaseHostInvalid
}
return fmt.Sprintf("postgres://%s:%s@%s:%s/%s?sslmode=%s",
url.QueryEscape(dbConfig.DatabaseUser), url.QueryEscape(dbConfig.DatabasePassword), host, port, dbConfig.DatabaseName, dbConfig.DatabaseSSLMode), nil
}
+67
View File
@@ -0,0 +1,67 @@
package datastore
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
func TestGetMysqlConnectionString_TCP(t *testing.T) {
expectedValue := "username:password@tcp(1.2.3.4:3306)/dbname?charset=utf8mb4&parseTime=true"
actualValue, err := getMysqlConnectionString(&settings.DatabaseConfig{
DatabaseType: "mysql",
DatabaseHost: "1.2.3.4:3306",
DatabaseName: "dbname",
DatabaseUser: "username",
DatabasePassword: "password",
})
assert.Nil(t, err)
assert.Equal(t, expectedValue, actualValue)
}
func TestGetMysqlConnectionString_UnixSocket(t *testing.T) {
expectedValue := "username:password@unix(/path/to/mysql.sock)/dbname?charset=utf8mb4&parseTime=true"
actualValue, err := getMysqlConnectionString(&settings.DatabaseConfig{
DatabaseType: "mysql",
DatabaseHost: "/path/to/mysql.sock",
DatabaseName: "dbname",
DatabaseUser: "username",
DatabasePassword: "password",
})
assert.Nil(t, err)
assert.Equal(t, expectedValue, actualValue)
}
func TestGetPostgreSQLConnectionString_TCP(t *testing.T) {
expectedValue := "postgres://username:password@1.2.3.4:5432/dbname?sslmode=disable"
actualValue, err := getPostgresConnectionString(&settings.DatabaseConfig{
DatabaseType: "postgres",
DatabaseHost: "1.2.3.4:5432",
DatabaseName: "dbname",
DatabaseUser: "username",
DatabasePassword: "password",
DatabaseSSLMode: "disable",
})
assert.Nil(t, err)
assert.Equal(t, expectedValue, actualValue)
}
func TestGetPostgreSQLConnectionString_UnixSocket(t *testing.T) {
expectedValue := "postgres:///dbname?sslmode=disable&host=/path/to/postgres.sock&user=username&password=password"
actualValue, err := getPostgresConnectionString(&settings.DatabaseConfig{
DatabaseType: "postgres",
DatabaseHost: "/path/to/postgres.sock",
DatabaseName: "dbname",
DatabaseUser: "username",
DatabasePassword: "password",
DatabaseSSLMode: "disable",
})
assert.Nil(t, err)
assert.Equal(t, expectedValue, actualValue)
}
+1
View File
@@ -30,4 +30,5 @@ var (
ErrInvalidAmountExpression = NewNormalError(NormalSubcategoryConverter, 23, http.StatusBadRequest, "invalid amount expression")
ErrInvalidXmlFile = NewNormalError(NormalSubcategoryConverter, 24, http.StatusBadRequest, "invalid xml file")
ErrInvalidMT940File = NewNormalError(NormalSubcategoryConverter, 25, http.StatusBadRequest, "invalid mt940 file")
ErrInvalidJSONFile = NewNormalError(NormalSubcategoryConverter, 26, http.StatusBadRequest, "invalid json file")
)
+1
View File
@@ -40,6 +40,7 @@ const (
NormalSubcategoryConverter = 12
NormalSubcategoryUserCustomExchangeRate = 13
NormalSubcategoryModelContextProtocol = 14
NormalSubcategoryLargeLanguageModel = 15
)
// Error represents the specific error returned to user
+12
View File
@@ -0,0 +1,12 @@
package errs
import "net/http"
// Error codes related to large language model features
var (
ErrLargeLanguageModelProviderNotEnabled = NewNormalError(NormalSubcategoryLargeLanguageModel, 0, http.StatusBadRequest, "llm provider is not enabled")
ErrNoAIRecognitionImage = NewNormalError(NormalSubcategoryLargeLanguageModel, 1, http.StatusBadRequest, "no image for AI recognition")
ErrAIRecognitionImageIsEmpty = NewNormalError(NormalSubcategoryLargeLanguageModel, 2, http.StatusBadRequest, "image for AI recognition is empty")
ErrExceedMaxAIRecognitionImageFileSize = NewNormalError(NormalSubcategoryLargeLanguageModel, 3, http.StatusBadRequest, "exceed the maximum size of image file for AI recognition")
ErrNoTransactionInformationInImage = NewNormalError(NormalSubcategoryLargeLanguageModel, 4, http.StatusBadRequest, "no transaction information detected")
)
+2
View File
@@ -24,4 +24,6 @@ var (
ErrInvalidPasswordResetTokenExpiredTime = NewSystemError(SystemSubcategorySetting, 17, http.StatusInternalServerError, "invalid password reset token expired time")
ErrInvalidExchangeRatesDataSource = NewSystemError(SystemSubcategorySetting, 18, http.StatusInternalServerError, "invalid exchange rates data source")
ErrInvalidIpAddressPattern = NewSystemError(SystemSubcategorySetting, 19, http.StatusInternalServerError, "invalid ip address pattern")
ErrInvalidLLMProvider = NewSystemError(SystemSubcategorySetting, 20, http.StatusInternalServerError, "invalid llm provider")
ErrInvalidLLMModelId = NewSystemError(SystemSubcategorySetting, 21, http.StatusInternalServerError, "invalid llm model id")
)
@@ -2,7 +2,6 @@ package exchangerates
import (
"crypto/tls"
"fmt"
"io"
"net/http"
"sort"
@@ -25,13 +24,13 @@ type HttpExchangeRatesDataSource interface {
Parse(c core.Context, content []byte) (*models.LatestExchangeRateResponse, error)
}
// CommonHttpExchangeRatesDataSource defines the structure of common http exchange rates data source
type CommonHttpExchangeRatesDataSource struct {
ExchangeRatesDataSource
// CommonHttpExchangeRatesDataProvider defines the structure of common http exchange rates data provider
type CommonHttpExchangeRatesDataProvider struct {
ExchangeRatesDataProvider
dataSource HttpExchangeRatesDataSource
}
func (e *CommonHttpExchangeRatesDataSource) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
func (e *CommonHttpExchangeRatesDataProvider) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
transport := http.DefaultTransport.(*http.Transport).Clone()
utils.SetProxyUrl(transport, currentConfig.ExchangeRatesProxy)
@@ -49,7 +48,7 @@ func (e *CommonHttpExchangeRatesDataSource) GetLatestExchangeRates(c core.Contex
requests, err := e.dataSource.BuildRequests()
if err != nil {
log.Errorf(c, "[http_exchange_rates_datasource.GetLatestExchangeRates] failed to build requests for user \"uid:%d\", because %s", uid, err.Error())
log.Errorf(c, "[common_http_exchange_rates_data_provider.GetLatestExchangeRates] failed to build requests for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
@@ -59,7 +58,7 @@ func (e *CommonHttpExchangeRatesDataSource) GetLatestExchangeRates(c core.Contex
req := requests[i]
if len(req.Header.Values("User-Agent")) < 1 {
req.Header.Set("User-Agent", fmt.Sprintf("ezBookkeeping/%s", settings.Version))
req.Header.Set("User-Agent", settings.GetUserAgent())
} else if req.Header.Get("User-Agent") == "" {
req.Header.Del("User-Agent")
}
@@ -67,24 +66,24 @@ func (e *CommonHttpExchangeRatesDataSource) GetLatestExchangeRates(c core.Contex
resp, err := client.Do(req)
if err != nil {
log.Errorf(c, "[http_exchange_rates_datasource.GetLatestExchangeRates] failed to request latest exchange rate data for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
if resp.StatusCode != 200 {
log.Errorf(c, "[http_exchange_rates_datasource.GetLatestExchangeRates] failed to get latest exchange rate data response for user \"uid:%d\", because response code is not 200", uid)
log.Errorf(c, "[common_http_exchange_rates_data_provider.GetLatestExchangeRates] failed to request latest exchange rate data for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
log.Debugf(c, "[http_exchange_rates_datasource.GetLatestExchangeRates] response#%d is %s", i, body)
log.Debugf(c, "[common_http_exchange_rates_data_provider.GetLatestExchangeRates] response#%d is %s", i, body)
if resp.StatusCode != 200 {
log.Errorf(c, "[common_http_exchange_rates_data_provider.GetLatestExchangeRates] failed to get latest exchange rate data response for user \"uid:%d\", because response code is not %d", uid, resp.StatusCode)
return nil, errs.ErrFailedToRequestRemoteApi
}
exchangeRateResp, err := e.dataSource.Parse(c, body)
if err != nil {
log.Errorf(c, "[http_exchange_rates_datasource.GetLatestExchangeRates] failed to parse response for user \"uid:%d\", because %s", uid, err.Error())
log.Errorf(c, "[common_http_exchange_rates_data_provider.GetLatestExchangeRates] failed to parse response for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrFailedToRequestRemoteApi)
}
@@ -126,8 +125,8 @@ func (e *CommonHttpExchangeRatesDataSource) GetLatestExchangeRates(c core.Contex
return finalExchangeRateResponse, nil
}
func newCommonHttpExchangeRatesDataSource(dataSource HttpExchangeRatesDataSource) *CommonHttpExchangeRatesDataSource {
return &CommonHttpExchangeRatesDataSource{
func newCommonHttpExchangeRatesDataProvider(dataSource HttpExchangeRatesDataSource) *CommonHttpExchangeRatesDataProvider {
return &CommonHttpExchangeRatesDataProvider{
dataSource: dataSource,
}
}
@@ -6,8 +6,8 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// ExchangeRatesDataSource defines the structure of exchange rates data source
type ExchangeRatesDataSource interface {
// ExchangeRatesDataProvider defines the structure of exchange rates data provider
type ExchangeRatesDataProvider interface {
// GetLatestExchangeRates returns the common response entities
GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error)
}
@@ -7,71 +7,71 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// ExchangeRatesDataSourceContainer contains the current exchange rates data source
type ExchangeRatesDataSourceContainer struct {
current ExchangeRatesDataSource
// ExchangeRatesDataProviderContainer contains the current exchange rates data provider
type ExchangeRatesDataProviderContainer struct {
current ExchangeRatesDataProvider
}
// Initialize a exchange rates data source container singleton instance
// Initialize a exchange rates data provider container singleton instance
var (
Container = &ExchangeRatesDataSourceContainer{}
Container = &ExchangeRatesDataProviderContainer{}
)
// InitializeExchangeRatesDataSource initializes the current exchange rates data source according to the config
func InitializeExchangeRatesDataSource(config *settings.Config) error {
if config.ExchangeRatesDataSource == settings.ReserveBankOfAustraliaDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&ReserveBankOfAustraliaDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&ReserveBankOfAustraliaDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.BankOfCanadaDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&BankOfCanadaDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&BankOfCanadaDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.CzechNationalBankDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&CzechNationalBankDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&CzechNationalBankDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.DanmarksNationalbankDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&DanmarksNationalbankDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&DanmarksNationalbankDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.EuroCentralBankDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&EuroCentralBankDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&EuroCentralBankDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.NationalBankOfGeorgiaDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&NationalBankOfGeorgiaDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&NationalBankOfGeorgiaDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.CentralBankOfHungaryDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&CentralBankOfHungaryDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&CentralBankOfHungaryDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.BankOfIsraelDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&BankOfIsraelDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&BankOfIsraelDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.CentralBankOfMyanmarDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&CentralBankOfMyanmarDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&CentralBankOfMyanmarDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.NorgesBankDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&NorgesBankDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&NorgesBankDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.NationalBankOfPolandDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&NationalBankOfPolandDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&NationalBankOfPolandDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.NationalBankOfRomaniaDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&NationalBankOfRomaniaDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&NationalBankOfRomaniaDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.BankOfRussiaDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&BankOfRussiaDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&BankOfRussiaDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.SwissNationalBankDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&SwissNationalBankDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&SwissNationalBankDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.NationalBankOfUkraineDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&NationalBankOfUkraineDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&NationalBankOfUkraineDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.CentralBankOfUzbekistanDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&CentralBankOfUzbekistanDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&CentralBankOfUzbekistanDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.InternationalMonetaryFundDataSource {
Container.current = newCommonHttpExchangeRatesDataSource(&InternationalMonetaryFundDataSource{})
Container.current = newCommonHttpExchangeRatesDataProvider(&InternationalMonetaryFundDataSource{})
return nil
} else if config.ExchangeRatesDataSource == settings.UserCustomExchangeRatesDataSource {
Container.current = newUserCustomExchangeRatesDataSource()
Container.current = newUserCustomExchangeRatesDataProvider()
return nil
}
@@ -79,7 +79,7 @@ func InitializeExchangeRatesDataSource(config *settings.Config) error {
}
// GetLatestExchangeRates returns the latest exchange rates data from the current exchange rates data source
func (e *ExchangeRatesDataSourceContainer) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
func (e *ExchangeRatesDataProviderContainer) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
if Container.current == nil {
return nil, errs.ErrInvalidExchangeRatesDataSource
}
@@ -15,25 +15,25 @@ import (
const userDataSourceType = "user_custom"
// UserCustomExchangeRatesDataSource defines the structure of user custom exchange rates data source
type UserCustomExchangeRatesDataSource struct {
ExchangeRatesDataSource
// UserCustomExchangeRatesDataProvider defines the structure of user custom exchange rates data provider
type UserCustomExchangeRatesDataProvider struct {
ExchangeRatesDataProvider
users *services.UserService
userCustomExchangeRates *services.UserCustomExchangeRatesService
}
func (e *UserCustomExchangeRatesDataSource) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
func (e *UserCustomExchangeRatesDataProvider) GetLatestExchangeRates(c core.Context, uid int64, currentConfig *settings.Config) (*models.LatestExchangeRateResponse, error) {
user, err := e.users.GetUserById(c, uid)
if err != nil {
log.Errorf(c, "[user_custom_datasource.GetLatestExchangeRates] failed to get user \"uid:%d\", because %s", uid, err.Error())
log.Errorf(c, "[user_custom_data_provider.GetLatestExchangeRates] failed to get user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
customExchangeRates, err := e.userCustomExchangeRates.GetAllCustomExchangeRatesByUid(c, uid)
if err != nil {
log.Errorf(c, "[user_custom_datasource.GetLatestExchangeRates] failed to get user custom exchange rates for user \"uid:%d\", because %s", uid, err.Error())
log.Errorf(c, "[user_custom_data_provider.GetLatestExchangeRates] failed to get user custom exchange rates for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
@@ -93,8 +93,8 @@ func (e *UserCustomExchangeRatesDataSource) GetLatestExchangeRates(c core.Contex
return finalExchangeRateResponse, nil
}
func newUserCustomExchangeRatesDataSource() *UserCustomExchangeRatesDataSource {
return &UserCustomExchangeRatesDataSource{
func newUserCustomExchangeRatesDataProvider() *UserCustomExchangeRatesDataProvider {
return &UserCustomExchangeRatesDataProvider{
users: services.Users,
userCustomExchangeRates: services.UserCustomExchangeRates,
}
+34
View File
@@ -0,0 +1,34 @@
package data
import "reflect"
type LargeLanguageModelRequestPromptType byte
// Large Language Model Request Prompt Type
const (
LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_TEXT LargeLanguageModelRequestPromptType = 0
LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL LargeLanguageModelRequestPromptType = 1
)
type LargeLanguageModelResponseFormat byte
// Large Language Model Response Format
const (
LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_TEXT LargeLanguageModelResponseFormat = 0
LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON LargeLanguageModelResponseFormat = 1
)
// LargeLanguageModelRequest represents a request to a large language model
type LargeLanguageModelRequest struct {
Stream bool
SystemPrompt string
UserPrompt []byte
UserPromptType LargeLanguageModelRequestPromptType
UserPromptContentType string
ResponseJsonObjectType reflect.Type
}
// LargeLanguageModelTextualResponse represents a textual response from a large language model
type LargeLanguageModelTextualResponse struct {
Content string
}
@@ -0,0 +1,64 @@
package llm
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/googleai"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/ollama"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/openai"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// LargeLanguageModelProviderContainer contains the current large language model provider
type LargeLanguageModelProviderContainer struct {
receiptImageRecognitionCurrentProvider provider.LargeLanguageModelProvider
}
// Initialize a large language model provider container singleton instance
var (
Container = &LargeLanguageModelProviderContainer{}
)
// InitializeLargeLanguageModelProvider initializes the current large language model provider according to the config
func InitializeLargeLanguageModelProvider(config *settings.Config) error {
var err error = nil
if config.ReceiptImageRecognitionLLMConfig != nil {
Container.receiptImageRecognitionCurrentProvider, err = initializeLargeLanguageModelProvider(config.ReceiptImageRecognitionLLMConfig)
if err != nil {
return err
}
}
return nil
}
func initializeLargeLanguageModelProvider(llmConfig *settings.LLMConfig) (provider.LargeLanguageModelProvider, error) {
if llmConfig.LLMProvider == settings.OpenAILLMProvider {
return openai.NewOpenAILargeLanguageModelProvider(llmConfig), nil
} else if llmConfig.LLMProvider == settings.OpenAICompatibleLLMProvider {
return openai.NewOpenAICompatibleLargeLanguageModelProvider(llmConfig), nil
} else if llmConfig.LLMProvider == settings.OpenRouterLLMProvider {
return openai.NewOpenRouterLargeLanguageModelProvider(llmConfig), nil
} else if llmConfig.LLMProvider == settings.OllamaLLMProvider {
return ollama.NewOllamaLargeLanguageModelProvider(llmConfig), nil
} else if llmConfig.LLMProvider == settings.GoogleAILLMProvider {
return googleai.NewGoogleAILargeLanguageModelProvider(llmConfig), nil
} else if llmConfig.LLMProvider == "" {
return nil, nil
}
return nil, errs.ErrInvalidLLMProvider
}
// GetJsonResponseByReceiptImageRecognitionModel returns the json response from the current large language model provider by receipt image recognition model
func (l *LargeLanguageModelProviderContainer) GetJsonResponseByReceiptImageRecognitionModel(c core.Context, uid int64, currentConfig *settings.Config, request *data.LargeLanguageModelRequest) (*data.LargeLanguageModelTextualResponse, error) {
if currentConfig.ReceiptImageRecognitionLLMConfig == nil || Container.receiptImageRecognitionCurrentProvider == nil {
return nil, errs.ErrInvalidLLMProvider
}
return l.receiptImageRecognitionCurrentProvider.GetJsonResponse(c, uid, currentConfig.ReceiptImageRecognitionLLMConfig, request)
}
@@ -0,0 +1,102 @@
package common
import (
"crypto/tls"
"io"
"net/http"
"strings"
"time"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/settings"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
// HttpLargeLanguageModelAdapter defines the structure of http large language model adapter
type HttpLargeLanguageModelAdapter interface {
// BuildTextualRequest returns the http request by the provider api definition
BuildTextualRequest(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) (*http.Request, error)
// ParseTextualResponse returns the textual response entity by the provider api definition
ParseTextualResponse(c core.Context, uid int64, body []byte, responseType data.LargeLanguageModelResponseFormat) (*data.LargeLanguageModelTextualResponse, error)
}
// CommonHttpLargeLanguageModelProvider defines the structure of common http large language model provider
type CommonHttpLargeLanguageModelProvider struct {
provider.LargeLanguageModelProvider
adapter HttpLargeLanguageModelAdapter
}
// GetJsonResponse returns the json response from common http large language model provider
func (p *CommonHttpLargeLanguageModelProvider) GetJsonResponse(c core.Context, uid int64, currentLLMConfig *settings.LLMConfig, request *data.LargeLanguageModelRequest) (*data.LargeLanguageModelTextualResponse, error) {
response, err := p.getTextualResponse(c, uid, currentLLMConfig, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
if err != nil {
return nil, err
}
if strings.HasPrefix(response.Content, "```json") && strings.HasSuffix(response.Content, "```") {
response.Content = strings.TrimPrefix(response.Content, "```json")
response.Content = strings.TrimSuffix(response.Content, "```")
} else if strings.HasPrefix(response.Content, "```") && strings.HasSuffix(response.Content, "```") {
response.Content = strings.TrimPrefix(response.Content, "```")
response.Content = strings.TrimSuffix(response.Content, "```")
}
return response, nil
}
func (p *CommonHttpLargeLanguageModelProvider) getTextualResponse(c core.Context, uid int64, currentLLMConfig *settings.LLMConfig, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) (*data.LargeLanguageModelTextualResponse, error) {
transport := http.DefaultTransport.(*http.Transport).Clone()
utils.SetProxyUrl(transport, currentLLMConfig.LargeLanguageModelAPIProxy)
if currentLLMConfig.LargeLanguageModelAPISkipTLSVerify {
transport.TLSClientConfig = &tls.Config{
InsecureSkipVerify: true,
}
}
client := &http.Client{
Transport: transport,
Timeout: time.Duration(currentLLMConfig.LargeLanguageModelAPIRequestTimeout) * time.Millisecond,
}
httpRequest, err := p.adapter.BuildTextualRequest(c, uid, request, responseType)
if err != nil {
log.Errorf(c, "[common_http_large_language_model_provider.getTextualResponse] failed to build requests for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
httpRequest.Header.Set("User-Agent", settings.GetUserAgent())
resp, err := client.Do(httpRequest)
if err != nil {
log.Errorf(c, "[common_http_large_language_model_provider.getTextualResponse] failed to request large language model api for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
log.Debugf(c, "[common_http_large_language_model_provider.getTextualResponse] response is %s", body)
if resp.StatusCode != 200 {
log.Errorf(c, "[common_http_large_language_model_provider.getTextualResponse] failed to get large language model api response for user \"uid:%d\", because response code is %d", uid, resp.StatusCode)
return nil, errs.ErrFailedToRequestRemoteApi
}
return p.adapter.ParseTextualResponse(c, uid, body, responseType)
}
// NewCommonHttpLargeLanguageModelProvider creates a http adapter based large language model provider instance
func NewCommonHttpLargeLanguageModelProvider(adapter HttpLargeLanguageModelAdapter) *CommonHttpLargeLanguageModelProvider {
return &CommonHttpLargeLanguageModelProvider{
adapter: adapter,
}
}
@@ -0,0 +1,167 @@
package googleai
import (
"bytes"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/common"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
const googleAIGenerateContentAPIFormat = "https://generativelanguage.googleapis.com/v1beta/models/%s:generateContent"
// GoogleAILargeLanguageModelAdapter defines the structure of Google AI large language model adapter
type GoogleAILargeLanguageModelAdapter struct {
common.HttpLargeLanguageModelAdapter
GoogleAIAPIKey string
GoogleAIModelID string
}
// GoogleAIGenerateContentRequest defines the structure of Google AI generate content request
type GoogleAIGenerateContentRequest struct {
Contents []*GoogleAIGenerateContentRequestContent `json:"contents"`
}
// GoogleAIGenerateContentRequestContent defines the structure of Google AI generate content request content
type GoogleAIGenerateContentRequestContent struct {
Parts []*GoogleAIGenerateContentRequestContentPart `json:"parts"`
}
// GoogleAIGenerateContentRequestContentPart defines the structure of Google AI generate content request content part
type GoogleAIGenerateContentRequestContentPart struct {
Text string `json:"text,omitempty"`
InlineData *GoogleAIGenerateContentRequestInlineData `json:"inlineData,omitempty"`
}
// GoogleAIGenerateContentRequestInlineData defines the structure of Google AI generate content request inline data
type GoogleAIGenerateContentRequestInlineData struct {
MimeType string `json:"mimeType"`
Data string `json:"data"`
}
// GoogleAIGenerateContentResponse defines the structure of Google AI generate content response
type GoogleAIGenerateContentResponse struct {
Candidates []*GoogleAIGenerateContentResponseCandidate `json:"candidates"`
}
// GoogleAIGenerateContentResponseCandidate defines the structure of Google AI generate content response candidate
type GoogleAIGenerateContentResponseCandidate struct {
Content *GoogleAIGenerateContentResponseContent `json:"content"`
}
// GoogleAIGenerateContentResponseContent defines the structure of Google AI generate content response content
type GoogleAIGenerateContentResponseContent struct {
Part []*GoogleAIGenerateContentResponseContentPart `json:"parts"`
}
// GoogleAIGenerateContentResponseContentPart defines the structure of Google AI generate content response content part
type GoogleAIGenerateContentResponseContentPart struct {
Text *string `json:"text"`
}
// BuildTextualRequest returns the http request by Google AI large language model adapter
func (p *GoogleAILargeLanguageModelAdapter) BuildTextualRequest(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) (*http.Request, error) {
requestBody, err := p.buildJsonRequestBody(c, uid, request, responseType)
if err != nil {
return nil, err
}
requestUrl := fmt.Sprintf(googleAIGenerateContentAPIFormat, p.GoogleAIModelID)
httpRequest, err := http.NewRequest("POST", requestUrl, bytes.NewReader(requestBody))
if err != nil {
return nil, err
}
httpRequest.Header.Set("Content-Type", "application/json")
httpRequest.Header.Set("X-goog-api-key", p.GoogleAIAPIKey)
return httpRequest, nil
}
// ParseTextualResponse returns the textual response by Google AI large language model adapter
func (p *GoogleAILargeLanguageModelAdapter) ParseTextualResponse(c core.Context, uid int64, body []byte, responseType data.LargeLanguageModelResponseFormat) (*data.LargeLanguageModelTextualResponse, error) {
generateContentResponse := &GoogleAIGenerateContentResponse{}
err := json.Unmarshal(body, &generateContentResponse)
if err != nil {
log.Errorf(c, "[google_ai_large_language_model_adapter.ParseTextualResponse] failed to parse generate content response for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
if generateContentResponse == nil || generateContentResponse.Candidates == nil || len(generateContentResponse.Candidates) < 1 ||
generateContentResponse.Candidates[0].Content == nil || len(generateContentResponse.Candidates[0].Content.Part) < 1 ||
generateContentResponse.Candidates[0].Content.Part[0].Text == nil {
log.Errorf(c, "[google_ai_large_language_model_adapter.ParseTextualResponse] generate content response is invalid for user \"uid:%d\"", uid)
return nil, errs.ErrFailedToRequestRemoteApi
}
textualResponse := &data.LargeLanguageModelTextualResponse{
Content: *generateContentResponse.Candidates[0].Content.Part[0].Text,
}
return textualResponse, nil
}
func (p *GoogleAILargeLanguageModelAdapter) buildJsonRequestBody(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) ([]byte, error) {
if p.GoogleAIModelID == "" {
return nil, errs.ErrInvalidLLMModelId
}
generateContentRequest := &GoogleAIGenerateContentRequest{
Contents: []*GoogleAIGenerateContentRequestContent{
{
Parts: make([]*GoogleAIGenerateContentRequestContentPart, 0, 2),
},
},
}
if request.SystemPrompt != "" {
generateContentRequest.Contents[0].Parts = append(generateContentRequest.Contents[0].Parts, &GoogleAIGenerateContentRequestContentPart{
Text: request.SystemPrompt,
})
}
if len(request.UserPrompt) > 0 {
if request.UserPromptType == data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL {
imageBase64Data := base64.StdEncoding.EncodeToString(request.UserPrompt)
generateContentRequest.Contents[0].Parts = append(generateContentRequest.Contents[0].Parts, &GoogleAIGenerateContentRequestContentPart{
InlineData: &GoogleAIGenerateContentRequestInlineData{
MimeType: request.UserPromptContentType,
Data: imageBase64Data,
},
})
} else {
generateContentRequest.Contents[0].Parts = append(generateContentRequest.Contents[0].Parts, &GoogleAIGenerateContentRequestContentPart{
Text: string(request.UserPrompt),
})
}
}
requestBodyBytes, err := json.Marshal(generateContentRequest)
if err != nil {
log.Errorf(c, "[google_ai_large_language_model_adapter.buildJsonRequestBody] failed to marshal request body for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrOperationFailed
}
log.Debugf(c, "[google_ai_large_language_model_adapter.buildJsonRequestBody] request body is %s", requestBodyBytes)
return requestBodyBytes, nil
}
// NewGoogleAILargeLanguageModelProvider creates a new Google AI large language model provider instance
func NewGoogleAILargeLanguageModelProvider(llmConfig *settings.LLMConfig) provider.LargeLanguageModelProvider {
return common.NewCommonHttpLargeLanguageModelProvider(&GoogleAILargeLanguageModelAdapter{
GoogleAIAPIKey: llmConfig.GoogleAIAPIKey,
GoogleAIModelID: llmConfig.GoogleAIModelID,
})
}
@@ -0,0 +1,181 @@
package googleai
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
)
func TestGoogleAILargeLanguageModelAdapter_buildJsonRequestBody_TextualUserPrompt(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "You are a helpful assistant.",
UserPrompt: []byte("Hello, how are you?"),
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"contents\":[{\"parts\":[{\"text\":\"You are a helpful assistant.\"},{\"text\":\"Hello, how are you?\"}]}]}", string(bodyBytes))
}
func TestGoogleAILargeLanguageModelAdapter_buildJsonRequestBody_ImageUserPrompt(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "What's in this image?",
UserPrompt: []byte("fakedata"),
UserPromptType: data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL,
UserPromptContentType: "image/png",
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"contents\":[{\"parts\":[{\"text\":\"What's in this image?\"},{\"inlineData\":{\"mimeType\":\"image/png\",\"data\":\"ZmFrZWRhdGE=\"}}]}]}", string(bodyBytes))
}
func TestGoogleAILargeLanguageModelAdapter_ParseTextualResponse_ValidJsonResponse(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
response := `{
"responseId": "test-123",
"modelVersion": "test",
"usageMetadata": {
"promptTokenCount": 13,
"candidatesTokenCount": 7,
"totalTokenCount": 20
},
"candidates": [
{
"content": {
"parts": [
{
"text": "This is a test response"
}
]
},
"finish_reason": "stop",
"index": 0
}
]
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "This is a test response", result.Content)
}
func TestGoogleAILargeLanguageModelAdapter_ParseTextualResponse_EmptyResponse(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
response := `{
"responseId": "test-123",
"modelVersion": "test",
"usageMetadata": {
"promptTokenCount": 13,
"candidatesTokenCount": 7,
"totalTokenCount": 20
},
"candidates": [
{
"content": {
"parts": [
{
"text": ""
}
]
},
"finish_reason": "stop",
"index": 0
}
]
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "", result.Content)
}
func TestGoogleAILargeLanguageModelAdapter_ParseTextualResponse_EmptyCandidates(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
response := `{
"responseId": "test-123",
"modelVersion": "test",
"usageMetadata": {
"promptTokenCount": 13,
"candidatesTokenCount": 7,
"totalTokenCount": 20
},
"candidates": []
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestGoogleAILargeLanguageModelAdapter_ParseTextualResponse_NoPartText(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
response := `{
"responseId": "test-123",
"modelVersion": "test",
"usageMetadata": {
"promptTokenCount": 13,
"candidatesTokenCount": 7,
"totalTokenCount": 20
},
"candidates": [
{
"content": {
"parts": [
{
}
]
},
"finish_reason": "stop",
"index": 0
}
]
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestGoogleAILargeLanguageModelAdapter_ParseTextualResponse_InvalidJson(t *testing.T) {
adapter := &GoogleAILargeLanguageModelAdapter{
GoogleAIModelID: "test",
}
response := "error"
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
@@ -0,0 +1,13 @@
package provider
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// LargeLanguageModelProvider defines the structure of large language model provider
type LargeLanguageModelProvider interface {
// GetJsonResponse returns the json response from the large language model provider
GetJsonResponse(c core.Context, uid int64, currentLLMConfig *settings.LLMConfig, request *data.LargeLanguageModelRequest) (*data.LargeLanguageModelTextualResponse, error)
}
@@ -0,0 +1,166 @@
package ollama
import (
"bytes"
"encoding/base64"
"encoding/json"
"net/http"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/common"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
const ollamaChatCompletionsPath = "api/chat"
// OllamaLargeLanguageModelAdapter defines the structure of Ollama large language model adapter
type OllamaLargeLanguageModelAdapter struct {
common.HttpLargeLanguageModelAdapter
OllamaServerURL string
OllamaModelID string
}
// OllamaMessageRole defines the role of Ollama chat message
type OllamaMessageRole string
const (
OllamaMessageRoleSystem OllamaMessageRole = "system"
OllamaMessageRoleUser OllamaMessageRole = "user"
)
// OllamaChatRequest defines the structure of Ollama chat request
type OllamaChatRequest struct {
Model string `json:"model"`
Stream bool `json:"stream"`
Messages []*OllamaChatRequestMessage `json:"messages"`
Format string `json:"format,omitempty"`
}
// OllamaChatRequestMessage defines the structure of Ollama chat request message
type OllamaChatRequestMessage struct {
Role OllamaMessageRole `json:"role"`
Content string `json:"content"`
Images []string `json:"images,omitempty"`
}
// OllamaChatResponse defines the structure of Ollama chat response
type OllamaChatResponse struct {
Message *OllamaChatResponseMessage `json:"message"`
}
// OllamaChatResponseMessage defines the structure of Ollama chat response message
type OllamaChatResponseMessage struct {
Content *string `json:"content"`
}
// BuildTextualRequest returns the http request by Ollama large language model adapter
func (p *OllamaLargeLanguageModelAdapter) BuildTextualRequest(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) (*http.Request, error) {
requestBody, err := p.buildJsonRequestBody(c, uid, request, responseType)
if err != nil {
return nil, err
}
httpRequest, err := http.NewRequest("POST", p.getOllamaRequestUrl(), bytes.NewReader(requestBody))
if err != nil {
return nil, err
}
httpRequest.Header.Set("Content-Type", "application/json")
return httpRequest, nil
}
// ParseTextualResponse returns the textual response by Ollama large language model adapter
func (p *OllamaLargeLanguageModelAdapter) ParseTextualResponse(c core.Context, uid int64, body []byte, responseType data.LargeLanguageModelResponseFormat) (*data.LargeLanguageModelTextualResponse, error) {
chatResponse := &OllamaChatResponse{}
err := json.Unmarshal(body, &chatResponse)
if err != nil {
log.Errorf(c, "[ollama_large_language_model_adapter.ParseTextualResponse] failed to parse chat response for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
if chatResponse == nil || chatResponse.Message == nil || chatResponse.Message.Content == nil {
log.Errorf(c, "[ollama_large_language_model_adapter.ParseTextualResponse] chat response is invalid for user \"uid:%d\"", uid)
return nil, errs.ErrFailedToRequestRemoteApi
}
textualResponse := &data.LargeLanguageModelTextualResponse{
Content: *chatResponse.Message.Content,
}
return textualResponse, nil
}
func (p *OllamaLargeLanguageModelAdapter) buildJsonRequestBody(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) ([]byte, error) {
if p.OllamaModelID == "" {
return nil, errs.ErrInvalidLLMModelId
}
chatRequest := &OllamaChatRequest{
Model: p.OllamaModelID,
Stream: request.Stream,
Messages: make([]*OllamaChatRequestMessage, 0, 2),
}
if request.SystemPrompt != "" {
chatRequest.Messages = append(chatRequest.Messages, &OllamaChatRequestMessage{
Role: OllamaMessageRoleSystem,
Content: request.SystemPrompt,
})
}
if len(request.UserPrompt) > 0 {
if request.UserPromptType == data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL {
imageBase64Data := base64.StdEncoding.EncodeToString(request.UserPrompt)
chatRequest.Messages = append(chatRequest.Messages, &OllamaChatRequestMessage{
Role: OllamaMessageRoleUser,
Images: []string{imageBase64Data},
})
} else {
chatRequest.Messages = append(chatRequest.Messages, &OllamaChatRequestMessage{
Role: OllamaMessageRoleUser,
Content: string(request.UserPrompt),
})
}
}
if responseType == data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON {
chatRequest.Format = "json"
}
requestBodyBytes, err := json.Marshal(chatRequest)
if err != nil {
log.Errorf(c, "[ollama_large_language_model_adapter.buildJsonRequestBody] failed to marshal request body for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrOperationFailed
}
log.Debugf(c, "[ollama_large_language_model_adapter.buildJsonRequestBody] request body is %s", requestBodyBytes)
return requestBodyBytes, nil
}
func (p *OllamaLargeLanguageModelAdapter) getOllamaRequestUrl() string {
url := p.OllamaServerURL
if url[len(url)-1] != '/' {
url += "/"
}
url += ollamaChatCompletionsPath
return url
}
// NewOllamaLargeLanguageModelProvider creates a new Ollama large language model provider instance
func NewOllamaLargeLanguageModelProvider(llmConfig *settings.LLMConfig) provider.LargeLanguageModelProvider {
return common.NewCommonHttpLargeLanguageModelProvider(&OllamaLargeLanguageModelAdapter{
OllamaServerURL: llmConfig.OllamaServerURL,
OllamaModelID: llmConfig.OllamaModelID,
})
}
@@ -0,0 +1,143 @@
package ollama
import (
"encoding/json"
"testing"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
)
func TestOllamaLargeLanguageModelAdapter_buildJsonRequestBody_TextualUserPrompt(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{
OllamaModelID: "test",
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "You are a helpful assistant.",
UserPrompt: []byte("Hello, how are you?"),
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"model\":\"test\",\"stream\":false,\"messages\":[{\"role\":\"system\",\"content\":\"You are a helpful assistant.\"},{\"role\":\"user\",\"content\":\"Hello, how are you?\"}],\"format\":\"json\"}", string(bodyBytes))
}
func TestOllamaLargeLanguageModelAdapter_buildJsonRequestBody_ImageUserPrompt(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{
OllamaModelID: "test",
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "What's in this image?",
UserPrompt: []byte("fakedata"),
UserPromptType: data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL,
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"model\":\"test\",\"stream\":false,\"messages\":[{\"role\":\"system\",\"content\":\"What's in this image?\"},{\"role\":\"user\",\"content\":\"\",\"images\":[\"ZmFrZWRhdGE=\"]}],\"format\":\"json\"}", string(bodyBytes))
}
func TestOllamaLargeLanguageModelAdapter_ParseTextualResponse_ValidJsonResponse(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{}
response := `{
"model": "test",
"created_at": "2025-09-01T01:02:03.456789Z",
"message": {
"role": "assistant",
"content": "This is a test response"
}
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "This is a test response", result.Content)
}
func TestOllamaLargeLanguageModelAdapter_ParseTextualResponse_EmptyResponse(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{}
response := `{
"model": "test",
"created_at": "2025-09-01T01:02:03.456789Z",
"message": {
"role": "assistant",
"content": ""
}
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "", result.Content)
}
func TestOllamaLargeLanguageModelAdapter_ParseTextualResponse_EmptyMessage(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{}
response := `{
"model": "test",
"created_at": "2025-09-01T01:02:03.456789Z",
"message": {}
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestOllamaLargeLanguageModelAdapter_ParseTextualResponse_NoContentFieldInMessage(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{}
response := `{
"model": "test",
"created_at": "2025-09-01T01:02:03.456789Z",
"message": {
"role": "assistant"
}
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestOllamaLargeLanguageModelAdapter_ParseTextualResponse_InvalidJson(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{}
response := "error"
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestOllamaLargeLanguageModelAdapter_GetOllamaRequestUrl(t *testing.T) {
adapter := &OllamaLargeLanguageModelAdapter{
OllamaServerURL: "http://localhost:11434/",
}
url := adapter.getOllamaRequestUrl()
assert.Equal(t, "http://localhost:11434/api/chat", url)
adapter = &OllamaLargeLanguageModelAdapter{
OllamaServerURL: "http://localhost:11434",
}
url = adapter.getOllamaRequestUrl()
assert.Equal(t, "http://localhost:11434/api/chat", url)
adapter = &OllamaLargeLanguageModelAdapter{
OllamaServerURL: "http://example.com/ollama/",
}
url = adapter.getOllamaRequestUrl()
assert.Equal(t, "http://example.com/ollama/api/chat", url)
}
@@ -0,0 +1,44 @@
package openai
import (
"net/http"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// OpenAIOfficialChatCompletionsAPIProvider defines the structure of OpenAI official chat completions API provider
type OpenAIOfficialChatCompletionsAPIProvider struct {
OpenAIChatCompletionsAPIProvider
OpenAIAPIKey string
OpenAIModelID string
}
const openAIChatCompletionsUrl = "https://api.openai.com/v1/chat/completions"
// BuildChatCompletionsHttpRequest returns the chat completions http request by OpenAI official chat completions API provider
func (p *OpenAIOfficialChatCompletionsAPIProvider) BuildChatCompletionsHttpRequest(c core.Context, uid int64) (*http.Request, error) {
req, err := http.NewRequest("POST", openAIChatCompletionsUrl, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+p.OpenAIAPIKey)
return req, nil
}
// GetModelID returns the model id of OpenAI official chat completions API provider
func (p *OpenAIOfficialChatCompletionsAPIProvider) GetModelID() string {
return p.OpenAIModelID
}
// NewOpenAILargeLanguageModelProvider creates a new OpenAI large language model provider instance
func NewOpenAILargeLanguageModelProvider(llmConfig *settings.LLMConfig) provider.LargeLanguageModelProvider {
return newCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter(&OpenAIOfficialChatCompletionsAPIProvider{
OpenAIAPIKey: llmConfig.OpenAIAPIKey,
OpenAIModelID: llmConfig.OpenAIModelID,
})
}
@@ -0,0 +1,219 @@
package openai
import (
"bytes"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"github.com/invopop/jsonschema"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider/common"
"github.com/mayswind/ezbookkeeping/pkg/log"
)
// OpenAIChatCompletionsAPIProvider defines the structure of OpenAI chat completions API provider
type OpenAIChatCompletionsAPIProvider interface {
// BuildChatCompletionsHttpRequest returns the chat completions http request
BuildChatCompletionsHttpRequest(c core.Context, uid int64) (*http.Request, error)
// GetModelID returns the model id if supported, otherwise returns empty string
GetModelID() string
}
// CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter defines the structure of OpenAI common compatible large language model adapter based on chat completions api
type CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter struct {
common.HttpLargeLanguageModelAdapter
apiProvider OpenAIChatCompletionsAPIProvider
}
// OpenAIMessageRole defines the role of OpenAI chat completions message
type OpenAIMessageRole string
// OpenAI Message Roles
const (
OpenAIMessageRoleSystem OpenAIMessageRole = "system"
OpenAIMessageRoleUser OpenAIMessageRole = "user"
)
// OpenAIChatCompletionsRequestResponseFormatType defines the type of OpenAI chat completions request response format
type OpenAIChatCompletionsRequestResponseFormatType string
// OpenAI Chat Completions Request Response Format Types
const (
OpenAIChatCompletionsRequestResponseFormatTypeJsonObject OpenAIChatCompletionsRequestResponseFormatType = "json_object"
OpenAIChatCompletionsRequestResponseFormatTypeJsonSchema OpenAIChatCompletionsRequestResponseFormatType = "json_schema"
)
// OpenAIChatCompletionsRequest defines the structure of OpenAI chat completions request
type OpenAIChatCompletionsRequest struct {
Model string `json:"model"`
Stream bool `json:"stream"`
Messages []any `json:"messages"`
ResponseFormat *OpenAIChatCompletionsRequestResponseFormat `json:"response_format,omitempty"`
}
// OpenAIChatCompletionsRequestMessage defines the structure of OpenAI chat completions request message
type OpenAIChatCompletionsRequestMessage[T string | []*OpenAIChatCompletionsRequestImageContent] struct {
Role OpenAIMessageRole `json:"role"`
Content T `json:"content"`
}
// OpenAIChatCompletionsRequestImageContent defines the structure of OpenAI chat completions request image content
type OpenAIChatCompletionsRequestImageContent struct {
Type string `json:"type"`
ImageURL *OpenAIChatCompletionsRequestImageUrl `json:"image_url"`
}
// OpenAIChatCompletionsRequestResponseFormat defines the structure of OpenAI chat completions request response format
type OpenAIChatCompletionsRequestResponseFormat struct {
Type OpenAIChatCompletionsRequestResponseFormatType `json:"type"`
JsonSchema *jsonschema.Schema `json:"json_schema,omitempty"`
}
// OpenAIChatCompletionsRequestImageUrl defines the structure of OpenAI image url
type OpenAIChatCompletionsRequestImageUrl struct {
Url string `json:"url"`
}
// OpenAIChatCompletionsResponse defines the structure of OpenAI chat completions response
type OpenAIChatCompletionsResponse struct {
Choices []*OpenAIChatCompletionsResponseChoice `json:"choices"`
}
// OpenAIChatCompletionsResponseChoice defines the structure of OpenAI chat completions response choice
type OpenAIChatCompletionsResponseChoice struct {
Message *OpenAIChatCompletionsResponseMessage `json:"message"`
}
// OpenAIChatCompletionsResponseMessage defines the structure of OpenAI chat completions response message
type OpenAIChatCompletionsResponseMessage struct {
Content *string `json:"content"`
}
// BuildTextualRequest returns the http request by OpenAI common compatible adapter
func (p *CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter) BuildTextualRequest(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) (*http.Request, error) {
requestBody, err := p.buildJsonRequestBody(c, uid, request, responseType)
if err != nil {
return nil, err
}
httpRequest, err := p.apiProvider.BuildChatCompletionsHttpRequest(c, uid)
if err != nil {
return nil, err
}
httpRequest.Body = io.NopCloser(bytes.NewReader(requestBody))
httpRequest.Header.Set("Content-Type", "application/json")
return httpRequest, nil
}
// ParseTextualResponse returns the textual response by OpenAI common compatible adapter
func (p *CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter) ParseTextualResponse(c core.Context, uid int64, body []byte, responseType data.LargeLanguageModelResponseFormat) (*data.LargeLanguageModelTextualResponse, error) {
chatCompletionsResponse := &OpenAIChatCompletionsResponse{}
err := json.Unmarshal(body, &chatCompletionsResponse)
if err != nil {
log.Errorf(c, "[openai_common_compatible_large_language_model_adapter.ParseTextualResponse] failed to parse chat completions response for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrFailedToRequestRemoteApi
}
if chatCompletionsResponse == nil || chatCompletionsResponse.Choices == nil || len(chatCompletionsResponse.Choices) < 1 ||
chatCompletionsResponse.Choices[0].Message == nil ||
chatCompletionsResponse.Choices[0].Message.Content == nil {
log.Errorf(c, "[openai_common_compatible_large_language_model_adapter.ParseTextualResponse] chat completions response is invalid for user \"uid:%d\"", uid)
return nil, errs.ErrFailedToRequestRemoteApi
}
textualResponse := &data.LargeLanguageModelTextualResponse{
Content: *chatCompletionsResponse.Choices[0].Message.Content,
}
return textualResponse, nil
}
func (p *CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter) buildJsonRequestBody(c core.Context, uid int64, request *data.LargeLanguageModelRequest, responseType data.LargeLanguageModelResponseFormat) ([]byte, error) {
if p.apiProvider.GetModelID() == "" {
return nil, errs.ErrInvalidLLMModelId
}
chatCompletionsRequest := &OpenAIChatCompletionsRequest{
Model: p.apiProvider.GetModelID(),
Stream: request.Stream,
Messages: make([]any, 0, 2),
}
if request.SystemPrompt != "" {
chatCompletionsRequest.Messages = append(chatCompletionsRequest.Messages, &OpenAIChatCompletionsRequestMessage[string]{
Role: OpenAIMessageRoleSystem,
Content: request.SystemPrompt,
})
}
if len(request.UserPrompt) > 0 {
if request.UserPromptType == data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL {
imageBase64Data := "data:" + request.UserPromptContentType + ";base64," + base64.StdEncoding.EncodeToString(request.UserPrompt)
chatCompletionsRequest.Messages = append(chatCompletionsRequest.Messages, &OpenAIChatCompletionsRequestMessage[[]*OpenAIChatCompletionsRequestImageContent]{
Role: OpenAIMessageRoleUser,
Content: []*OpenAIChatCompletionsRequestImageContent{
{
Type: "image_url",
ImageURL: &OpenAIChatCompletionsRequestImageUrl{
Url: imageBase64Data,
},
},
},
})
} else {
chatCompletionsRequest.Messages = append(chatCompletionsRequest.Messages, &OpenAIChatCompletionsRequestMessage[string]{
Role: OpenAIMessageRoleUser,
Content: string(request.UserPrompt),
})
}
}
if responseType == data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON {
if request.ResponseJsonObjectType != nil {
schemeGenerator := jsonschema.Reflector{
Anonymous: true,
DoNotReference: true,
ExpandedStruct: true,
}
schema := schemeGenerator.ReflectFromType(request.ResponseJsonObjectType)
schema.Version = ""
chatCompletionsRequest.ResponseFormat = &OpenAIChatCompletionsRequestResponseFormat{
Type: OpenAIChatCompletionsRequestResponseFormatTypeJsonSchema,
JsonSchema: schema,
}
} else {
chatCompletionsRequest.ResponseFormat = &OpenAIChatCompletionsRequestResponseFormat{
Type: OpenAIChatCompletionsRequestResponseFormatTypeJsonObject,
}
}
}
requestBodyBytes, err := json.Marshal(chatCompletionsRequest)
if err != nil {
log.Errorf(c, "[openai_common_compatible_large_language_model_adapter.buildJsonRequestBody] failed to marshal request body for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.ErrOperationFailed
}
log.Debugf(c, "[openai_common_compatible_large_language_model_adapter.buildJsonRequestBody] request body is %s", requestBodyBytes)
return requestBodyBytes, nil
}
func newCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter(apiProvider OpenAIChatCompletionsAPIProvider) provider.LargeLanguageModelProvider {
return common.NewCommonHttpLargeLanguageModelProvider(&CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: apiProvider,
})
}
@@ -0,0 +1,163 @@
package openai
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/data"
)
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_buildJsonRequestBody_TextualUserPrompt(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{
OpenAIModelID: "test",
},
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "You are a helpful assistant.",
UserPrompt: []byte("Hello, how are you?"),
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"model\":\"test\",\"stream\":false,\"messages\":[{\"role\":\"system\",\"content\":\"You are a helpful assistant.\"},{\"role\":\"user\",\"content\":\"Hello, how are you?\"}],\"response_format\":{\"type\":\"json_object\"}}", string(bodyBytes))
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_buildJsonRequestBody_ImageUserPrompt(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{
OpenAIModelID: "test",
},
}
request := &data.LargeLanguageModelRequest{
SystemPrompt: "What's in this image?",
UserPrompt: []byte("fakedata"),
UserPromptType: data.LARGE_LANGUAGE_MODEL_REQUEST_PROMPT_TYPE_IMAGE_URL,
UserPromptContentType: "image/png",
}
bodyBytes, err := adapter.buildJsonRequestBody(core.NewNullContext(), 0, request, data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
var body map[string]interface{}
err = json.Unmarshal(bodyBytes, &body)
assert.Nil(t, err)
assert.Equal(t, "{\"model\":\"test\",\"stream\":false,\"messages\":[{\"role\":\"system\",\"content\":\"What's in this image?\"},{\"role\":\"user\",\"content\":[{\"type\":\"image_url\",\"image_url\":{\"url\":\"data:image/png;base64,ZmFrZWRhdGE=\"}}]}],\"response_format\":{\"type\":\"json_object\"}}", string(bodyBytes))
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_ParseTextualResponse_ValidJsonResponse(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{},
}
response := `{
"id": "test-123",
"object": "chat.completion",
"created": 1234567890,
"model": "test",
"usage": {
"prompt_tokens": 13,
"completion_tokens": 7,
"total_tokens": 20
},
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"role": "assistant",
"content": "This is a test response"
}
}
]
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "This is a test response", result.Content)
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_ParseTextualResponse_EmptyResponse(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{},
}
response := `{
"id": "test-123",
"object": "chat.completion",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"role": "assistant",
"content": ""
}
}
]
}`
result, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.Nil(t, err)
assert.Equal(t, "", result.Content)
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_ParseTextualResponse_EmptyChoices(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{},
}
response := `{
"id": "test-123",
"object": "chat.completion",
"choices": []
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_ParseTextualResponse_NoChoiceContent(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{},
}
response := `{
"id": "chatcmpl-123",
"object": "chat.completion",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"role": "assistant"
}
}
]
}`
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
func TestCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter_ParseTextualResponse_InvalidJson(t *testing.T) {
adapter := &CommonOpenAIChatCompletionsAPILargeLanguageModelAdapter{
apiProvider: &OpenAIOfficialChatCompletionsAPIProvider{},
}
response := "error"
_, err := adapter.ParseTextualResponse(core.NewNullContext(), 0, []byte(response), data.LARGE_LANGUAGE_MODEL_RESPONSE_FORMAT_JSON)
assert.EqualError(t, err, "failed to request third party api")
}
@@ -0,0 +1,59 @@
package openai
import (
"net/http"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
const openAICompatibleChatCompletionsPath = "chat/completions"
// OpenAICompatibleChatCompletionsAPIProvider defines the structure of OpenAI compatible chat completions API provider
type OpenAICompatibleChatCompletionsAPIProvider struct {
OpenAIChatCompletionsAPIProvider
OpenAICompatibleBaseURL string
OpenAICompatibleAPIKey string
OpenAICompatibleModelID string
}
// BuildChatCompletionsHttpRequest returns the chat completions http request by OpenAI compatible chat completions API provider
func (p *OpenAICompatibleChatCompletionsAPIProvider) BuildChatCompletionsHttpRequest(c core.Context, uid int64) (*http.Request, error) {
req, err := http.NewRequest("POST", p.getFinalChatCompletionsRequestUrl(), nil)
if err != nil {
return nil, err
}
if p.OpenAICompatibleAPIKey != "" {
req.Header.Set("Authorization", "Bearer "+p.OpenAICompatibleAPIKey)
}
return req, nil
}
// GetModelID returns the model id of OpenAI compatible chat completions API provider
func (p *OpenAICompatibleChatCompletionsAPIProvider) GetModelID() string {
return p.OpenAICompatibleModelID
}
func (p *OpenAICompatibleChatCompletionsAPIProvider) getFinalChatCompletionsRequestUrl() string {
url := p.OpenAICompatibleBaseURL
if url[len(url)-1] != '/' {
url += "/"
}
url += openAICompatibleChatCompletionsPath
return url
}
// NewOpenAICompatibleLargeLanguageModelProvider creates a new OpenAI compatible large language model provider instance
func NewOpenAICompatibleLargeLanguageModelProvider(llmConfig *settings.LLMConfig) provider.LargeLanguageModelProvider {
return newCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter(&OpenAICompatibleChatCompletionsAPIProvider{
OpenAICompatibleBaseURL: llmConfig.OpenAICompatibleBaseURL,
OpenAICompatibleAPIKey: llmConfig.OpenAICompatibleAPIKey,
OpenAICompatibleModelID: llmConfig.OpenAICompatibleModelID,
})
}
@@ -0,0 +1,27 @@
package openai
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestOpenAICompatibleChatCompletionsAPIProvider_GetFinalRequestUrl(t *testing.T) {
apiProvider := &OpenAICompatibleChatCompletionsAPIProvider{
OpenAICompatibleBaseURL: "https://api.example.com/v1/",
}
url := apiProvider.getFinalChatCompletionsRequestUrl()
assert.Equal(t, "https://api.example.com/v1/chat/completions", url)
apiProvider = &OpenAICompatibleChatCompletionsAPIProvider{
OpenAICompatibleBaseURL: "https://api.example.com/v1",
}
url = apiProvider.getFinalChatCompletionsRequestUrl()
assert.Equal(t, "https://api.example.com/v1/chat/completions", url)
apiProvider = &OpenAICompatibleChatCompletionsAPIProvider{
OpenAICompatibleBaseURL: "https://example.com/api",
}
url = apiProvider.getFinalChatCompletionsRequestUrl()
assert.Equal(t, "https://example.com/api/chat/completions", url)
}
@@ -0,0 +1,46 @@
package openai
import (
"net/http"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/llm/provider"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// OpenRouterChatCompletionsAPIProvider defines the structure of OpenRouter chat completions API provider
type OpenRouterChatCompletionsAPIProvider struct {
OpenAIChatCompletionsAPIProvider
OpenRouterAPIKey string
OpenRouterModelID string
}
const openRouterChatCompletionsUrl = "https://openrouter.ai/api/v1/chat/completions"
// BuildChatCompletionsHttpRequest returns the chat completions http request by OpenRouter chat completions API provider
func (p *OpenRouterChatCompletionsAPIProvider) BuildChatCompletionsHttpRequest(c core.Context, uid int64) (*http.Request, error) {
req, err := http.NewRequest("POST", openRouterChatCompletionsUrl, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+p.OpenRouterAPIKey)
req.Header.Set("HTTP-Referer", "https://ezbookkeeping.mayswind.net/")
req.Header.Set("X-Title", "ezBookkeeping")
return req, nil
}
// GetModelID returns the model id of OpenRouter chat completions API provider
func (p *OpenRouterChatCompletionsAPIProvider) GetModelID() string {
return p.OpenRouterModelID
}
// NewOpenRouterLargeLanguageModelProvider creates a new OpenRouter large language model provider instance
func NewOpenRouterLargeLanguageModelProvider(llmConfig *settings.LLMConfig) provider.LargeLanguageModelProvider {
return newCommonOpenAIChatCompletionsAPILargeLanguageModelAdapter(&OpenRouterChatCompletionsAPIProvider{
OpenRouterAPIKey: llmConfig.OpenRouterAPIKey,
OpenRouterModelID: llmConfig.OpenRouterModelID,
})
}
+6
View File
@@ -15,6 +15,9 @@ var AllLanguages = map[string]*LocaleInfo{
"es": {
Content: es,
},
"fr": {
Content: fr,
},
"it": {
Content: it,
},
@@ -30,6 +33,9 @@ var AllLanguages = map[string]*LocaleInfo{
"ru": {
Content: ru,
},
"th": {
Content: th,
},
"uk": {
Content: uk,
},
+30
View File
@@ -0,0 +1,30 @@
package locales
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
)
var fr = &LocaleTextItems{
DefaultTypes: &DefaultTypes{
DecimalSeparator: core.DECIMAL_SEPARATOR_COMMA,
DigitGroupingSymbol: core.DIGIT_GROUPING_SYMBOL_SPACE,
},
DataConverterTextItems: &DataConverterTextItems{
Alipay: "Alipay",
WeChatWallet: "Wallet",
},
VerifyEmailTextItems: &VerifyEmailTextItems{
Title: "Vérifier l'e-mail",
SalutationFormat: "Bonjour %s,",
DescriptionAboveBtn: "Cliquez sur le lien ci-dessous pour confirmer votre adresse e-mail.",
VerifyEmail: "Vérifier l'e-mail",
DescriptionBelowBtnFormat: "Si vous n'avez pas créé de compte %s, vous pouvez ignorer cet e-mail. Si vous ne pouvez pas cliquer sur le lien ci-dessus, copiez l'URL ci-dessus et collez-la dans votre navigateur. Le lien de vérification expire après %v minutes.",
},
ForgetPasswordMailTextItems: &ForgetPasswordMailTextItems{
Title: "Réinitialiser le mot de passe",
SalutationFormat: "Bonjour %s,",
DescriptionAboveBtn: "Nous avons récemment reçu une demande de réinitialisation de votre mot de passe. Cliquez sur le lien ci-dessous pour réinitialiser votre mot de passe.",
ResetPassword: "Réinitialiser le mot de passe",
DescriptionBelowBtnFormat: "Si vous n'avez pas demandé la réinitialisation de votre mot de passe, vous pouvez ignorer cet e-mail. Si vous ne pouvez pas cliquer sur le lien ci-dessus, copiez l'URL ci-dessus et collez-la dans votre navigateur. Le lien de réinitialisation du mot de passe expire après %v minutes.",
},
}
+30
View File
@@ -0,0 +1,30 @@
package locales
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
)
var th = &LocaleTextItems{
DefaultTypes: &DefaultTypes{
DecimalSeparator: core.DECIMAL_SEPARATOR_DOT,
DigitGroupingSymbol: core.DIGIT_GROUPING_SYMBOL_COMMA,
},
DataConverterTextItems: &DataConverterTextItems{
Alipay: "Alipay",
WeChatWallet: "Wallet",
},
VerifyEmailTextItems: &VerifyEmailTextItems{
Title: "ยืนยันอีเมล",
SalutationFormat: "สวัสดี %s,",
DescriptionAboveBtn: "โปรดคลิกที่ลิงค์ด้านล่างเพื่อยืนยันที่อยู่อีเมลของคุณ",
VerifyEmail: "ยืนยันอีเมล",
DescriptionBelowBtnFormat: "หากคุณไม่ได้ลงทะเบียนสำหรับบัญชี %s โปรดละเว้นอีเมลนี้ หากคุณไม่สามารถคลิกลิงก์ด้านบน โปรดคัดลอก URL ด้านบนและวางลงในเบราว์เซอร์ของคุณ ลิงก์ยืนยันอีเมลจะหมดอายุหลังจาก %v นาที",
},
ForgetPasswordMailTextItems: &ForgetPasswordMailTextItems{
Title: "รีเซ็ตรหัสผ่านใหม่",
SalutationFormat: "สวัสดี %s,",
DescriptionAboveBtn: "เมื่อเร็ว ๆ นี้เราได้รับการร้องขอให้รีเซ็ตรหัสผ่านของคุณ คุณสามารถคลิกลิงก์ด้านล่างเพื่อรีเซ็ตรหัสผ่านของคุณ",
ResetPassword: "ตั้งรหัสผ่านใหม่",
DescriptionBelowBtnFormat: "หากคุณไม่ได้ร้องขอให้รีเซ็ตรหัสผ่าน โปรดละเว้นอีเมลนี้ หากคุณไม่สามารถคลิกลิงก์ด้านบน โปรดคัดลอก URL ด้านบนและวางลงในเบราว์เซอร์ของคุณ ลิงก์รีเซ็ตรหัสผ่านจะหมดอายุหลังจาก %v นาที",
},
}
+35 -10
View File
@@ -121,10 +121,30 @@ func (h *mcpAddTransactionToolHandler) Handle(c *core.WebContext, callToolReq *M
return nil, nil, err
}
categoriesMap := services.GetTransactionCategoryService().GetVisibleCategoryNameMapByList(allCategories)
category, exists := categoriesMap[addTransactionRequest.SecondaryCategoryName]
var transactionCategory *models.TransactionCategory = nil
if !exists {
for i := 0; i < len(allCategories); i++ {
category := allCategories[i]
if category.Hidden || category.ParentCategoryId == models.LevelOneTransactionCategoryParentId {
continue
}
if category.Name == addTransactionRequest.SecondaryCategoryName {
if category.Type == models.CATEGORY_TYPE_INCOME && addTransactionRequest.Type == transactionTypeIncome {
transactionCategory = category
break
} else if category.Type == models.CATEGORY_TYPE_EXPENSE && addTransactionRequest.Type == transactionTypeExpense {
transactionCategory = category
break
} else if category.Type == models.CATEGORY_TYPE_TRANSFER && addTransactionRequest.Type == transactionTypeTransfer {
transactionCategory = category
break
}
}
}
if transactionCategory == nil {
log.Warnf(c, "[add_transaction.Handle] secondary category \"%s\" not found for user \"uid:%d\"", addTransactionRequest.SecondaryCategoryName, uid)
return nil, nil, errs.ErrTransactionCategoryNotFound
}
@@ -139,7 +159,7 @@ func (h *mcpAddTransactionToolHandler) Handle(c *core.WebContext, callToolReq *M
return nil, nil, err
}
tagMaps := services.GetTransactionTagService().GetTagNameMapByList(allTags)
tagMaps := services.GetTransactionTagService().GetVisibleTagNameMapByList(allTags)
tagIds = make([]int64, 0, len(addTransactionRequest.Tags))
for _, tagName := range addTransactionRequest.Tags {
@@ -151,7 +171,12 @@ func (h *mcpAddTransactionToolHandler) Handle(c *core.WebContext, callToolReq *M
}
}
transaction := h.createNewTransactionModel(uid, &addTransactionRequest, category.CategoryId, sourceAccount.AccountId, destinationAccountId, c.ClientIP())
transaction, err := h.createNewTransactionModel(uid, &addTransactionRequest, transactionCategory.CategoryId, sourceAccount.AccountId, destinationAccountId, c.ClientIP())
if err != nil {
return nil, nil, err
}
transactionEditable := user.CanEditTransactionByTransactionTime(transaction.TransactionTime, transaction.TimezoneUtcOffset)
if !transactionEditable {
@@ -212,7 +237,7 @@ func (h *mcpAddTransactionToolHandler) Handle(c *core.WebContext, callToolReq *M
}
}
func (h *mcpAddTransactionToolHandler) createNewTransactionModel(uid int64, addTransactionRequest *MCPAddTransactionRequest, categoryId int64, sourceAccountId int64, destinationAccountId int64, clientIp string) *models.Transaction {
func (h *mcpAddTransactionToolHandler) createNewTransactionModel(uid int64, addTransactionRequest *MCPAddTransactionRequest, categoryId int64, sourceAccountId int64, destinationAccountId int64, clientIp string) (*models.Transaction, error) {
var transactionDbType models.TransactionDbType
if addTransactionRequest.Type == transactionTypeExpense {
@@ -226,13 +251,13 @@ func (h *mcpAddTransactionToolHandler) createNewTransactionModel(uid int64, addT
transactionTime, err := utils.ParseFromLongDateTimeWithTimezoneRFC3339Format(addTransactionRequest.Time)
if err != nil {
return nil
return nil, err
}
amount, err := utils.ParseAmount(addTransactionRequest.Amount)
if err != nil {
return nil
return nil, err
}
transaction := &models.Transaction{
@@ -254,13 +279,13 @@ func (h *mcpAddTransactionToolHandler) createNewTransactionModel(uid int64, addT
destinationAmount, err := utils.ParseAmount(addTransactionRequest.DestinationAmount)
if err != nil {
return nil
return nil, err
}
transaction.RelatedAccountAmount = destinationAmount
}
return transaction
return transaction, nil
}
func (h *mcpAddTransactionToolHandler) createNewMCPAddTransactionResponse(c *core.WebContext, transaction *models.Transaction, accountsMap map[int64]*models.Account, dryRun bool) (any, []*MCPTextContent, error) {
+1 -1
View File
@@ -122,7 +122,7 @@ func handleTool[T MCPTextContent | MCPImageContent | MCPAudioContent | MCPResour
IsError: false,
}
if ctx.GetHeader(MCPProtocolVersionHeaderName) > string(ToolResultStructuredContentMinVersion) {
if ctx.GetHeader(MCPProtocolVersionHeaderName) >= string(ToolResultStructuredContentMinVersion) {
callToolResp.StructuredContent = structuredResponse
}
@@ -49,10 +49,14 @@ func (h *mcpQueryAllTransactionTagsToolHandler) Handle(c *core.WebContext, callT
return nil, nil, err
}
tagNames := make([]string, len(tags))
tagNames := make([]string, 0, len(tags))
for i := 0; i < len(tags); i++ {
tagNames[i] = tags[i].Name
if tags[i].Hidden {
continue
}
tagNames = append(tagNames, tags[i].Name)
}
response := MCPAllQueryTransactionTagsResponse{
+7 -9
View File
@@ -19,7 +19,7 @@ type MCPQueryTransactionsRequest struct {
StartTime string `json:"start_time" jsonschema:"format=date-time" jsonschema_description:"Start time for the query in RFC 3339 format (e.g. 2023-01-01T12:00:00Z)"`
EndTime string `json:"end_time" jsonschema:"format=date-time" jsonschema_description:"End time for the query in RFC 3339 format or (e.g. 2023-01-01T12:00:00Z)"`
Type string `json:"type,omitempty" jsonschema:"enum=income,enum=expense,enum=transfer" jsonschema_description:"Transaction type to filter by (income, expense, transfer) (optional)"`
SecondaryCategoryName string `json:"category_name,omitempty" jsonschema_description:"Secondary category name to filter transactions by (optional)"`
SecondaryCategoryName string `json:"category_name,omitempty" jsonschema_description:"Primary or secondary category name to filter transactions by (optional)"`
AccountName string `json:"account_name,omitempty" jsonschema_description:"Account name to filter transactions by (optional)"`
Keyword string `json:"keyword,omitempty" jsonschema_description:"Keyword to search in transaction description (optional)"`
Count int32 `json:"count,omitempty" jsonschema:"default=100" jsonschema_description:"Maximum number of results to return (default: 100)"`
@@ -126,13 +126,12 @@ func (h *mcpQueryTransactionsToolHandler) Handle(c *core.WebContext, callToolReq
return nil, nil, err
}
accountsMap := services.GetAccountService().GetVisibleAccountNameMapByList(allAccounts)
filterAccountIds := make([]int64, 0)
if queryTransactionsRequest.AccountName != "" {
if account, exists := accountsMap[queryTransactionsRequest.AccountName]; exists {
filterAccountIds = append(filterAccountIds, account.AccountId)
} else {
filterAccountIds = services.GetAccountService().GetAccountOrSubAccountIdsByAccountName(allAccounts, queryTransactionsRequest.AccountName)
if len(filterAccountIds) < 1 {
return nil, nil, errs.ErrAccountNotFound
}
}
@@ -144,13 +143,12 @@ func (h *mcpQueryTransactionsToolHandler) Handle(c *core.WebContext, callToolReq
return nil, nil, err
}
categoriesMap := services.GetTransactionCategoryService().GetVisibleCategoryNameMapByList(allCategories)
filterCategoryIds := make([]int64, 0)
if queryTransactionsRequest.SecondaryCategoryName != "" {
if category, exists := categoriesMap[queryTransactionsRequest.SecondaryCategoryName]; exists {
filterCategoryIds = append(filterCategoryIds, category.CategoryId)
} else {
filterCategoryIds = services.GetTransactionCategoryService().GetCategoryOrSubCategoryIdsByCategoryName(allCategories, queryTransactionsRequest.SecondaryCategoryName)
if len(filterCategoryIds) < 1 {
return nil, nil, errs.ErrTransactionCategoryNotFound
}
}
+6
View File
@@ -5,6 +5,12 @@ type ClearDataRequest struct {
Password string `json:"password" binding:"omitempty,min=6,max=128"`
}
// ClearAccountTransactionsRequest represents all parameters of clear transaction data of a specific account request
type ClearAccountTransactionsRequest struct {
AccountId int64 `json:"accountId,string" binding:"required,min=1"`
Password string `json:"password" binding:"omitempty,min=6,max=128"`
}
// DataStatisticsResponse represents a view-object of user data statistic
type DataStatisticsResponse struct {
TotalAccountCount int64 `json:"totalAccountCount,string"`
+20
View File
@@ -14,6 +14,26 @@ type ImportTransaction struct {
OriginalTagNames []string
}
// ImportTransactionRequest represents all parameters of the imported transaction data
type ImportTransactionRequest struct {
Transactions []*ImportTransactionRequestItem
}
// ImportTransactionRequestItem represents a single item of the imported transaction data
type ImportTransactionRequestItem struct {
Time string `json:"time"`
UtcOffset string `json:"utcOffset"`
Type string `json:"type"`
CategoryName string `json:"categoryName,omitempty"`
SourceAccountName string `json:"sourceAccountName,omitempty"`
DestinationAccountName string `json:"destinationAccountName,omitempty"`
SourceAmount string `json:"sourceAmount"`
DestinationAmount string `json:"destinationAmount,omitempty"`
GeoLocation string `json:"geoLocation,omitempty"`
TagNames string `json:"tagNames,omitempty"`
Comment string `json:"comment,omitempty"`
}
// ImportTransactionResponse represents a view-object of the imported transaction data
type ImportTransactionResponse struct {
Type TransactionType `json:"type"`
+27
View File
@@ -0,0 +1,27 @@
package models
// RecognizedReceiptImageResponse represents a view-object of recognized receipt image response
type RecognizedReceiptImageResponse struct {
Type TransactionType `json:"type"`
Time int64 `json:"time,omitempty"`
CategoryId int64 `json:"categoryId,string,omitempty"`
SourceAccountId int64 `json:"sourceAccountId,string,omitempty"`
DestinationAccountId int64 `json:"destinationAccountId,string,omitempty"`
SourceAmount int64 `json:"sourceAmount,omitempty"`
DestinationAmount int64 `json:"destinationAmount,omitempty"`
TagIds []string `json:"tagIds,omitempty"`
Comment string `json:"comment,omitempty"`
}
// RecognizedReceiptImageResult represents the result of recognized receipt image
type RecognizedReceiptImageResult struct {
Type string `json:"type,omitempty" jsonschema:"enum=income,enum=expense,enum=transfer" jsonschema_description:"Transaction type (income, expense, transfer)"`
Time string `json:"time" jsonschema:"format=date-time" jsonschema_description:"Transaction time in long date time format (YYYY-MM-DD HH:mm:ss, e.g. 2023-01-01 12:00:00)"`
Amount string `json:"amount,omitempty" jsonschema_description:"Transaction amount"`
AccountName string `json:"account,omitempty" jsonschema_description:"Account name for the transaction"`
CategoryName string `json:"category,omitempty" jsonschema_description:"Category name for the transaction"`
TagNames []string `json:"tags,omitempty" jsonschema_description:"List of tags associated with the transaction (maximum 10 tags allowed)"`
Description string `json:"description,omitempty" jsonschema_description:"Transaction description"`
DestinationAmount string `json:"destination_amount,omitempty" jsonschema_description:"Destination amount for transfer transactions"`
DestinationAccountName string `json:"destination_account,omitempty" jsonschema_description:"Destination account name for transfer transactions"`
}
+2
View File
@@ -258,6 +258,8 @@ type TransactionStatisticTrendsRequest struct {
// TransactionAmountsRequest represents all parameters of transaction amounts request
type TransactionAmountsRequest struct {
Query string `form:"query"`
ExcludeAccountIds string `form:"exclude_account_ids"`
ExcludeCategoryIds string `form:"exclude_category_ids"`
UseTransactionTimezone bool `form:"use_transaction_timezone"`
}
+2 -2
View File
@@ -199,8 +199,8 @@ type UserProfileUpdateRequest struct {
DefaultCurrency string `json:"defaultCurrency" binding:"omitempty,len=3,validCurrency"`
FirstDayOfWeek *core.WeekDay `json:"firstDayOfWeek" binding:"omitempty,min=0,max=6"`
FiscalYearStart *core.FiscalYearStart `json:"fiscalYearStart" binding:"omitempty,validFiscalYearStart"`
CalendarDisplayType *core.CalendarDisplayType `json:"calendarDisplayType" binding:"omitempty,min=0,max=2"`
DateDisplayType *core.DateDisplayType `json:"dateDisplayType" binding:"omitempty,min=0,max=2"`
CalendarDisplayType *core.CalendarDisplayType `json:"calendarDisplayType" binding:"omitempty,min=0,max=4"`
DateDisplayType *core.DateDisplayType `json:"dateDisplayType" binding:"omitempty,min=0,max=3"`
LongDateFormat *core.LongDateFormat `json:"longDateFormat" binding:"omitempty,min=0,max=3"`
ShortDateFormat *core.ShortDateFormat `json:"shortDateFormat" binding:"omitempty,min=0,max=3"`
LongTimeFormat *core.LongTimeFormat `json:"longTimeFormat" binding:"omitempty,min=0,max=3"`
+4 -2
View File
@@ -17,8 +17,10 @@ var ALL_ALLOWED_CLOUD_SYNC_APP_SETTING_KEY_TYPES = map[string]UserApplicationClo
// Basic Settings
"showAccountBalance": USER_APPLICATION_CLOUD_SETTING_TYPE_BOOLEAN,
// Overview Page
"showAmountInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_BOOLEAN,
"timezoneUsedForStatisticsInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_NUMBER,
"showAmountInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_BOOLEAN,
"timezoneUsedForStatisticsInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_NUMBER,
"overviewAccountFilterInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_STRING_BOOLEAN_MAP,
"overviewTransactionCategoryFilterInHomePage": USER_APPLICATION_CLOUD_SETTING_TYPE_STRING_BOOLEAN_MAP,
// Transaction List Page
"itemsCountInTransactionListPage": USER_APPLICATION_CLOUD_SETTING_TYPE_NUMBER,
"showTotalAmountInTransactionListPage": USER_APPLICATION_CLOUD_SETTING_TYPE_BOOLEAN,
+43 -2
View File
@@ -698,7 +698,7 @@ func (s *AccountService) DeleteAccount(c core.Context, uid int64, accountId int6
}
}
transactionTemplateQueryCondition := fmt.Sprintf("uid=? AND deleted=? AND (template_type=? || (template_type=? && scheduled_frequency_type<>? && (scheduled_end_time IS NULL OR scheduled_end_time>=?))) AND (account_id IN (%s) OR related_account_id IN (%s))", accountAndSubAccountIdsConditions.String(), accountAndSubAccountIdsConditions.String())
transactionTemplateQueryCondition := fmt.Sprintf("uid=? AND deleted=? AND (template_type=? OR (template_type=? AND scheduled_frequency_type<>? AND (scheduled_end_time IS NULL OR scheduled_end_time>=?))) AND (account_id IN (%s) OR related_account_id IN (%s))", accountAndSubAccountIdsConditions.String(), accountAndSubAccountIdsConditions.String())
transactionTemplateQueryConditionParams := make([]any, 0, len(accountAndSubAccountIds)*2+6)
transactionTemplateQueryConditionParams = append(transactionTemplateQueryConditionParams, uid)
transactionTemplateQueryConditionParams = append(transactionTemplateQueryConditionParams, false)
@@ -804,7 +804,7 @@ func (s *AccountService) DeleteSubAccount(c core.Context, uid int64, accountId i
}
}
exists, err := sess.Cols("uid", "deleted", "account_id", "related_account_id", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? || (template_type=? && scheduled_frequency_type<>? && (scheduled_end_time IS NULL OR scheduled_end_time>=?))) AND (account_id=? OR related_account_id=?)", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now, accountId, accountId).Limit(1).Exist(&models.TransactionTemplate{})
exists, err := sess.Cols("uid", "deleted", "account_id", "related_account_id", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? OR (template_type=? AND scheduled_frequency_type<>? AND (scheduled_end_time IS NULL OR scheduled_end_time>=?))) AND (account_id=? OR related_account_id=?)", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now, accountId, accountId).Limit(1).Exist(&models.TransactionTemplate{})
if err != nil {
return err
@@ -940,3 +940,44 @@ func (s *AccountService) GetAccountOrSubAccountIds(c core.Context, accountIds st
return allAccountIds, nil
}
// GetAccountOrSubAccountIdsByAccountName returns a list of account ids or sub-account ids according to given account name
func (s *AccountService) GetAccountOrSubAccountIdsByAccountName(accounts []*models.Account, accountName string) []int64 {
accountIds := make([]int64, 0)
parentAccountIds := make([]int64, 0)
childAccountByParentAccountId := make(map[int64][]*models.Account)
for i := 0; i < len(accounts); i++ {
account := accounts[i]
if account.Name == accountName {
if account.Type == models.ACCOUNT_TYPE_SINGLE_ACCOUNT {
accountIds = append(accountIds, account.AccountId)
} else if account.Type == models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS {
parentAccountIds = append(parentAccountIds, account.AccountId)
}
} else if account.ParentAccountId > 0 {
childAccounts, exists := childAccountByParentAccountId[account.ParentAccountId]
if !exists {
childAccounts = make([]*models.Account, 0)
}
childAccounts = append(childAccounts, account)
childAccountByParentAccountId[account.ParentAccountId] = childAccounts
}
}
for i := 0; i < len(parentAccountIds); i++ {
parentAccountId := parentAccountIds[i]
if childAccounts, exists := childAccountByParentAccountId[parentAccountId]; exists {
for j := 0; j < len(childAccounts); j++ {
childAccount := childAccounts[j]
accountIds = append(accountIds, childAccount.AccountId)
}
}
}
return accountIds
}
+218
View File
@@ -0,0 +1,218 @@
package services
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
func TestGetAccountMapByList_EmptyList(t *testing.T) {
accounts := make([]*models.Account, 0)
actualAccountMap := Accounts.GetAccountMapByList(accounts)
assert.NotNil(t, actualAccountMap)
assert.Equal(t, 0, len(actualAccountMap))
}
func TestGetAccountMapByList_MultipleList(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Cash Account",
Category: models.ACCOUNT_CATEGORY_CASH,
},
{
AccountId: 1002,
Name: "Checking Account",
Category: models.ACCOUNT_CATEGORY_CHECKING_ACCOUNT,
},
{
AccountId: 1003,
Name: "Credit Card",
Category: models.ACCOUNT_CATEGORY_CREDIT_CARD,
},
}
actualAccountMap := Accounts.GetAccountMapByList(accounts)
assert.Equal(t, 3, len(actualAccountMap))
assert.Contains(t, actualAccountMap, int64(1001))
assert.Contains(t, actualAccountMap, int64(1002))
assert.Contains(t, actualAccountMap, int64(1003))
assert.Equal(t, "Cash Account", actualAccountMap[1001].Name)
assert.Equal(t, "Checking Account", actualAccountMap[1002].Name)
assert.Equal(t, "Credit Card", actualAccountMap[1003].Name)
}
func TestGetVisibleAccountNameMapByList_EmptyList(t *testing.T) {
accounts := make([]*models.Account, 0)
actualAccountMap := Accounts.GetVisibleAccountNameMapByList(accounts)
assert.NotNil(t, actualAccountMap)
assert.Equal(t, 0, len(actualAccountMap))
}
func TestGetVisibleAccountNameMapByList_WithHiddenAccount(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Visible Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
Hidden: false,
},
{
AccountId: 1002,
Name: "Hidden Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
Hidden: true,
},
}
actualAccountMap := Accounts.GetVisibleAccountNameMapByList(accounts)
assert.Equal(t, 1, len(actualAccountMap))
assert.Contains(t, actualAccountMap, "Visible Account")
assert.NotContains(t, actualAccountMap, "Hidden Account")
}
func TestGetVisibleAccountNameMapByList_WithParentAccount(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Single Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
Hidden: false,
},
{
AccountId: 1002,
Name: "Multi Sub Accounts",
Type: models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS,
Hidden: false,
},
}
actualAccountMap := Accounts.GetVisibleAccountNameMapByList(accounts)
assert.Equal(t, 1, len(actualAccountMap))
assert.Contains(t, actualAccountMap, "Single Account")
assert.NotContains(t, actualAccountMap, "Multi Sub Accounts")
}
func TestGetAccountNames_EmptyList(t *testing.T) {
accounts := make([]*models.Account, 0)
actualAccountMap := Accounts.GetAccountNames(accounts)
assert.NotNil(t, actualAccountMap)
assert.Equal(t, 0, len(actualAccountMap))
}
func TestGetAccountNames_MultipleList(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Cash Account",
},
{
AccountId: 1002,
Name: "Checking Account",
},
{
AccountId: 1003,
Name: "Credit Card",
},
}
actualAccountMap := Accounts.GetAccountNames(accounts)
assert.Equal(t, 3, len(actualAccountMap))
assert.Equal(t, "Cash Account", actualAccountMap[0])
assert.Equal(t, "Checking Account", actualAccountMap[1])
assert.Equal(t, "Credit Card", actualAccountMap[2])
}
func TestGetAccountOrSubAccountIdsByAccountName_EmptyList(t *testing.T) {
accounts := make([]*models.Account, 0)
actualAccountMap := Accounts.GetAccountOrSubAccountIdsByAccountName(accounts, "Test Account")
assert.NotNil(t, actualAccountMap)
assert.Equal(t, 0, len(actualAccountMap))
}
func TestGetAccountOrSubAccountIdsByAccountName_NotMatch(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Cash Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
},
}
actualAccountMap := Accounts.GetAccountOrSubAccountIdsByAccountName(accounts, "Non-existent Account")
assert.NotNil(t, actualAccountMap)
assert.Equal(t, 0, len(actualAccountMap))
}
func TestGetAccountOrSubAccountIdsByAccountName_MatchSingle(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Cash Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
},
}
actualAccountMap := Accounts.GetAccountOrSubAccountIdsByAccountName(accounts, "Cash Account")
assert.Equal(t, 1, len(actualAccountMap))
assert.Contains(t, actualAccountMap, int64(1001))
}
func TestGetAccountOrSubAccountIdsByAccountName_MatchMultiple(t *testing.T) {
accounts := []*models.Account{
{
AccountId: 1001,
Name: "Test Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
},
{
AccountId: 2001,
Name: "Test Account",
Type: models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS,
ParentAccountId: 0,
},
{
AccountId: 2002,
Name: "Sub 1-1",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
ParentAccountId: 2001,
},
{
AccountId: 2003,
Name: "Sub 1-2",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
ParentAccountId: 2001,
},
{
AccountId: 3001,
Name: "Test Account",
Type: models.ACCOUNT_TYPE_MULTI_SUB_ACCOUNTS,
ParentAccountId: 0,
},
{
AccountId: 3002,
Name: "Sub 2-1",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
ParentAccountId: 3001,
},
{
AccountId: 4001,
Name: "Other Account",
Type: models.ACCOUNT_TYPE_SINGLE_ACCOUNT,
},
}
actualAccountMap := Accounts.GetAccountOrSubAccountIdsByAccountName(accounts, "Test Account")
assert.Equal(t, 4, len(actualAccountMap))
assert.Contains(t, actualAccountMap, int64(1001))
assert.Contains(t, actualAccountMap, int64(2002))
assert.Contains(t, actualAccountMap, int64(2003))
assert.Contains(t, actualAccountMap, int64(3002))
assert.NotContains(t, actualAccountMap, int64(2001))
assert.NotContains(t, actualAccountMap, int64(3001))
assert.NotContains(t, actualAccountMap, int64(4001))
}
+42 -18
View File
@@ -397,7 +397,7 @@ func (s *TransactionCategoryService) DeleteCategory(c core.Context, uid int64, c
return errs.ErrTransactionCategoryInUseCannotBeDeleted
}
exists, err = sess.Cols("uid", "deleted", "category_id", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? || (template_type=? && scheduled_frequency_type<>? && (scheduled_end_time IS NULL OR scheduled_end_time>=?)))", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now).In("category_id", categoryAndSubCategoryIds).Limit(1).Exist(&models.TransactionTemplate{})
exists, err = sess.Cols("uid", "deleted", "category_id", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? OR (template_type=? AND scheduled_frequency_type<>? AND (scheduled_end_time IS NULL OR scheduled_end_time>=?)))", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now).In("category_id", categoryAndSubCategoryIds).Limit(1).Exist(&models.TransactionTemplate{})
if err != nil {
return err
@@ -522,23 +522,6 @@ func (s *TransactionCategoryService) GetVisibleSubCategoryNameMapByList(categori
return expenseCategoryMap, incomeCategoryMap, transferCategoryMap
}
// GetVisibleCategoryNameMapByList returns visible transaction category map by a list
func (s *TransactionCategoryService) GetVisibleCategoryNameMapByList(categories []*models.TransactionCategory) map[string]*models.TransactionCategory {
categoryMap := make(map[string]*models.TransactionCategory)
for i := 0; i < len(categories); i++ {
category := categories[i]
if category.Hidden {
continue
}
categoryMap[category.Name] = category
}
return categoryMap
}
// GetCategoryNames returns a list with transaction category names from transaction category models list
func (s *TransactionCategoryService) GetCategoryNames(categories []*models.TransactionCategory) []string {
categoryNames := make([]string, len(categories))
@@ -602,3 +585,44 @@ func (s *TransactionCategoryService) GetCategoryOrSubCategoryIds(c core.Context,
return allCategoryIds, nil
}
// GetCategoryOrSubCategoryIdsByCategoryName returns a list of transaction category ids or sub-category ids according to given category name
func (s *TransactionCategoryService) GetCategoryOrSubCategoryIdsByCategoryName(categories []*models.TransactionCategory, categoryName string) []int64 {
categoryIds := make([]int64, 0)
parentCategoryIds := make([]int64, 0)
childCategoryByParentCategoryId := make(map[int64][]*models.TransactionCategory)
for i := 0; i < len(categories); i++ {
category := categories[i]
if category.Name == categoryName {
if category.ParentCategoryId != models.LevelOneTransactionCategoryParentId {
categoryIds = append(categoryIds, category.CategoryId)
} else if category.ParentCategoryId == models.LevelOneTransactionCategoryParentId {
parentCategoryIds = append(parentCategoryIds, category.CategoryId)
}
} else if category.ParentCategoryId != models.LevelOneTransactionCategoryParentId {
childCategories, exists := childCategoryByParentCategoryId[category.ParentCategoryId]
if !exists {
childCategories = make([]*models.TransactionCategory, 0)
}
childCategories = append(childCategories, category)
childCategoryByParentCategoryId[category.ParentCategoryId] = childCategories
}
}
for i := 0; i < len(parentCategoryIds); i++ {
parentCategoryId := parentCategoryIds[i]
if childCategories, exists := childCategoryByParentCategoryId[parentCategoryId]; exists {
for j := 0; j < len(childCategories); j++ {
childCategory := childCategories[j]
categoryIds = append(categoryIds, childCategory.CategoryId)
}
}
}
return categoryIds
}
+301
View File
@@ -0,0 +1,301 @@
package services
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
func TestGetCategoryMapByList_EmptyList(t *testing.T) {
categories := make([]*models.TransactionCategory, 0)
actualCategoryMap := TransactionCategories.GetCategoryMapByList(categories)
assert.NotNil(t, actualCategoryMap)
assert.Equal(t, 0, len(actualCategoryMap))
}
func TestGetCategoryMapByList_MultipleCategories(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 1002,
Name: "Category Name2",
Type: models.CATEGORY_TYPE_INCOME,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 1003,
Name: "Category Name3",
Type: models.CATEGORY_TYPE_TRANSFER,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: true,
},
}
actualCategoryMap := TransactionCategories.GetCategoryMapByList(categories)
assert.Equal(t, 3, len(actualCategoryMap))
assert.Contains(t, actualCategoryMap, int64(1001))
assert.Contains(t, actualCategoryMap, int64(1002))
assert.Contains(t, actualCategoryMap, int64(1003))
assert.Equal(t, "Category Name", actualCategoryMap[1001].Name)
assert.Equal(t, "Category Name2", actualCategoryMap[1002].Name)
assert.Equal(t, "Category Name3", actualCategoryMap[1003].Name)
}
func TestGetVisibleSubCategoryNameMapByList_EmptyList(t *testing.T) {
categories := make([]*models.TransactionCategory, 0)
expenseCategoryMap, incomeCategoryMap, transferCategoryMap := TransactionCategories.GetVisibleSubCategoryNameMapByList(categories)
assert.NotNil(t, expenseCategoryMap)
assert.NotNil(t, incomeCategoryMap)
assert.NotNil(t, transferCategoryMap)
assert.Equal(t, 0, len(expenseCategoryMap))
assert.Equal(t, 0, len(incomeCategoryMap))
assert.Equal(t, 0, len(transferCategoryMap))
}
func TestGetVisibleSubCategoryNameMapByList_OnlyParentCategories(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 1002,
Name: "Category Name2",
Type: models.CATEGORY_TYPE_INCOME,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
}
expenseCategoryMap, incomeCategoryMap, transferCategoryMap := TransactionCategories.GetVisibleSubCategoryNameMapByList(categories)
assert.Equal(t, 0, len(expenseCategoryMap))
assert.Equal(t, 0, len(incomeCategoryMap))
assert.Equal(t, 0, len(transferCategoryMap))
}
func TestGetVisibleSubCategoryNameMapByList_WithHiddenCategories(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 2001,
Name: "Category Name2",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: 1001,
Hidden: true,
},
{
CategoryId: 2002,
Name: "Category Name3",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: 1001,
Hidden: false,
},
}
expenseCategoryMap, incomeCategoryMap, transferCategoryMap := TransactionCategories.GetVisibleSubCategoryNameMapByList(categories)
assert.Equal(t, 1, len(expenseCategoryMap))
assert.Contains(t, expenseCategoryMap, "Category Name3")
assert.NotContains(t, expenseCategoryMap, "Category Name2")
assert.Equal(t, 0, len(incomeCategoryMap))
assert.Equal(t, 0, len(transferCategoryMap))
}
func TestGetVisibleSubCategoryNameMapByList_AllTypes(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 2001,
Name: "Category Name2",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: 1001,
Hidden: false,
},
{
CategoryId: 1002,
Name: "Category Name3",
Type: models.CATEGORY_TYPE_INCOME,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 2002,
Name: "Category Name4",
Type: models.CATEGORY_TYPE_INCOME,
ParentCategoryId: 1002,
Hidden: false,
},
{
CategoryId: 1003,
Name: "Category Name5",
Type: models.CATEGORY_TYPE_TRANSFER,
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
Hidden: false,
},
{
CategoryId: 2003,
Name: "Category Name6",
Type: models.CATEGORY_TYPE_TRANSFER,
ParentCategoryId: 1003,
Hidden: false,
},
}
expenseCategoryMap, incomeCategoryMap, transferCategoryMap := TransactionCategories.GetVisibleSubCategoryNameMapByList(categories)
assert.Equal(t, 1, len(expenseCategoryMap))
assert.Contains(t, expenseCategoryMap, "Category Name2")
assert.Contains(t, expenseCategoryMap["Category Name2"], "Category Name")
assert.Equal(t, 1, len(incomeCategoryMap))
assert.Contains(t, incomeCategoryMap, "Category Name4")
assert.Contains(t, incomeCategoryMap["Category Name4"], "Category Name3")
assert.Equal(t, 1, len(transferCategoryMap))
assert.Contains(t, transferCategoryMap, "Category Name6")
assert.Contains(t, transferCategoryMap["Category Name6"], "Category Name5")
}
func TestGetVisibleSubCategoryNameMapByList_OrphanSubCategories(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 2001,
Name: "Category Name",
Type: models.CATEGORY_TYPE_EXPENSE,
ParentCategoryId: 9999,
Hidden: false,
},
}
expenseCategoryMap, incomeCategoryMap, transferCategoryMap := TransactionCategories.GetVisibleSubCategoryNameMapByList(categories)
assert.Equal(t, 0, len(expenseCategoryMap))
assert.Equal(t, 0, len(incomeCategoryMap))
assert.Equal(t, 0, len(transferCategoryMap))
}
func TestGetCategoryNames_EmptyList(t *testing.T) {
categories := make([]*models.TransactionCategory, 0)
actualNames := TransactionCategories.GetCategoryNames(categories)
assert.NotNil(t, actualNames)
assert.Equal(t, 0, len(actualNames))
}
func TestGetCategoryNames_MultipleCategories(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
},
{
CategoryId: 1002,
Name: "Category Name2",
},
{
CategoryId: 1003,
Name: "Category Name3",
},
}
actualNames := TransactionCategories.GetCategoryNames(categories)
assert.Equal(t, 3, len(actualNames))
assert.Equal(t, "Category Name", actualNames[0])
assert.Equal(t, "Category Name2", actualNames[1])
assert.Equal(t, "Category Name3", actualNames[2])
}
func TestGetCategoryOrSubCategoryIdsByCategoryName_EmptyList(t *testing.T) {
categories := make([]*models.TransactionCategory, 0)
actualIds := TransactionCategories.GetCategoryOrSubCategoryIdsByCategoryName(categories, "Category Name")
assert.NotNil(t, actualIds)
assert.Equal(t, 0, len(actualIds))
}
func TestGetCategoryOrSubCategoryIdsByCategoryName_NotExistName(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
},
}
actualIds := TransactionCategories.GetCategoryOrSubCategoryIdsByCategoryName(categories, "Non-existent Category")
assert.NotNil(t, actualIds)
assert.Equal(t, 0, len(actualIds))
}
func TestGetCategoryOrSubCategoryIdsByCategoryName_ParentCategoryWithoutChildren(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
},
}
actualIds := TransactionCategories.GetCategoryOrSubCategoryIdsByCategoryName(categories, "Category Name")
assert.NotNil(t, actualIds)
assert.Equal(t, 0, len(actualIds))
}
func TestGetCategoryOrSubCategoryIdsByCategoryName_BothParentAndSubCategory(t *testing.T) {
categories := []*models.TransactionCategory{
{
CategoryId: 1001,
Name: "Category Name",
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
},
{
CategoryId: 2001,
Name: "Category Name",
ParentCategoryId: 1001,
},
{
CategoryId: 2002,
Name: "Category Name2",
ParentCategoryId: 1001,
},
{
CategoryId: 1002,
Name: "Category Name3",
ParentCategoryId: models.LevelOneTransactionCategoryParentId,
},
{
CategoryId: 2003,
Name: "Category Name",
ParentCategoryId: 1002,
},
}
actualIds := TransactionCategories.GetCategoryOrSubCategoryIdsByCategoryName(categories, "Category Name")
assert.Equal(t, 3, len(actualIds))
assert.Contains(t, actualIds, int64(2001))
assert.Contains(t, actualIds, int64(2002))
assert.Contains(t, actualIds, int64(2003))
}
+9 -3
View File
@@ -397,7 +397,7 @@ func (s *TransactionTagService) DeleteTag(c core.Context, uid int64, tagId int64
}
var relatedTransactionTemplatesByTag []*models.TransactionTemplate
err = sess.Cols("uid", "deleted", "tag_ids", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? || (template_type=? && scheduled_frequency_type<>? && (scheduled_end_time IS NULL OR scheduled_end_time>=?))) && tag_ids LIKE ?", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now, "%%"+utils.Int64ToString(tagId)+"%%").Find(&relatedTransactionTemplatesByTag)
err = sess.Cols("uid", "deleted", "tag_ids", "template_type", "scheduled_frequency_type", "scheduled_end_time").Where("uid=? AND deleted=? AND (template_type=? OR (template_type=? AND scheduled_frequency_type<>? AND (scheduled_end_time IS NULL OR scheduled_end_time>=?))) AND tag_ids LIKE ?", uid, false, models.TRANSACTION_TEMPLATE_TYPE_NORMAL, models.TRANSACTION_TEMPLATE_TYPE_SCHEDULE, models.TRANSACTION_SCHEDULE_FREQUENCY_TYPE_DISABLED, now, "%%"+utils.Int64ToString(tagId)+"%%").Find(&relatedTransactionTemplatesByTag)
if err != nil {
return err
@@ -508,14 +508,20 @@ func (s *TransactionTagService) GetTagMapByList(tags []*models.TransactionTag) m
return tagMap
}
// GetTagNameMapByList returns a transaction tag map by a list
func (s *TransactionTagService) GetTagNameMapByList(tags []*models.TransactionTag) map[string]*models.TransactionTag {
// GetVisibleTagNameMapByList returns a visible transaction tag map by a list
func (s *TransactionTagService) GetVisibleTagNameMapByList(tags []*models.TransactionTag) map[string]*models.TransactionTag {
tagMap := make(map[string]*models.TransactionTag)
for i := 0; i < len(tags); i++ {
tag := tags[i]
if tag.Hidden {
continue
}
tagMap[tag.Name] = tag
}
return tagMap
}
+220
View File
@@ -0,0 +1,220 @@
package services
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
func TestGetTagMapByList_EmptyList(t *testing.T) {
tags := make([]*models.TransactionTag, 0)
actualTagMap := TransactionTags.GetTagMapByList(tags)
assert.NotNil(t, actualTagMap)
assert.Equal(t, 0, len(actualTagMap))
}
func TestGetTagMapByList_SingleTag(t *testing.T) {
tags := []*models.TransactionTag{
{
TagId: 1001,
Name: "Tag Name",
Hidden: false,
},
}
actualTagMap := TransactionTags.GetTagMapByList(tags)
assert.Equal(t, 1, len(actualTagMap))
assert.Contains(t, actualTagMap, int64(1001))
assert.Equal(t, "Tag Name", actualTagMap[1001].Name)
assert.Equal(t, false, actualTagMap[1001].Hidden)
}
func TestGetTagMapByList_MultipleTags(t *testing.T) {
tags := []*models.TransactionTag{
{
TagId: 1001,
Name: "Tag Name",
Hidden: false,
},
{
TagId: 1002,
Name: "Tag Name2",
Hidden: true,
},
{
TagId: 1003,
Name: "Tag Name3",
Hidden: false,
},
}
actualTagMap := TransactionTags.GetTagMapByList(tags)
assert.Equal(t, 3, len(actualTagMap))
assert.Contains(t, actualTagMap, int64(1001))
assert.Contains(t, actualTagMap, int64(1002))
assert.Contains(t, actualTagMap, int64(1003))
assert.Equal(t, "Tag Name", actualTagMap[1001].Name)
assert.Equal(t, "Tag Name2", actualTagMap[1002].Name)
assert.Equal(t, "Tag Name3", actualTagMap[1003].Name)
assert.Equal(t, false, actualTagMap[1001].Hidden)
assert.Equal(t, true, actualTagMap[1002].Hidden)
assert.Equal(t, false, actualTagMap[1003].Hidden)
}
func TestGetVisibleTagNameMapByList_EmptyList(t *testing.T) {
tags := make([]*models.TransactionTag, 0)
actualTagMap := TransactionTags.GetVisibleTagNameMapByList(tags)
assert.NotNil(t, actualTagMap)
assert.Equal(t, 0, len(actualTagMap))
}
func TestGetVisibleTagNameMapByList_MixedVisibilityTags(t *testing.T) {
tags := []*models.TransactionTag{
{
TagId: 1001,
Name: "Visible Tag",
Hidden: false,
},
{
TagId: 1002,
Name: "Hidden Tag",
Hidden: true,
},
{
TagId: 1003,
Name: "Visible Tag2",
Hidden: false,
},
}
actualTagMap := TransactionTags.GetVisibleTagNameMapByList(tags)
assert.Equal(t, 2, len(actualTagMap))
assert.Contains(t, actualTagMap, "Visible Tag")
assert.Contains(t, actualTagMap, "Visible Tag2")
assert.NotContains(t, actualTagMap, "Hidden Tag")
assert.Equal(t, int64(1001), actualTagMap["Visible Tag"].TagId)
assert.Equal(t, int64(1003), actualTagMap["Visible Tag2"].TagId)
}
func TestGetTagNames_EmptyList(t *testing.T) {
tags := make([]*models.TransactionTag, 0)
actualNames := TransactionTags.GetTagNames(tags)
assert.NotNil(t, actualNames)
assert.Equal(t, 0, len(actualNames))
}
func TestGetTagNames_MultipleTags(t *testing.T) {
tags := []*models.TransactionTag{
{
TagId: 1001,
Name: "Tag Name",
},
{
TagId: 1002,
Name: "Tag Name2",
},
{
TagId: 1003,
Name: "Tag Name3",
},
}
actualNames := TransactionTags.GetTagNames(tags)
assert.Equal(t, 3, len(actualNames))
assert.Equal(t, "Tag Name", actualNames[0])
assert.Equal(t, "Tag Name2", actualNames[1])
assert.Equal(t, "Tag Name3", actualNames[2])
}
func TestGetTagIds_EmptyString(t *testing.T) {
tagIds, err := TransactionTags.GetTagIds("")
assert.Nil(t, err)
assert.Nil(t, tagIds)
}
func TestGetTagIds_ZeroString(t *testing.T) {
tagIds, err := TransactionTags.GetTagIds("0")
assert.Nil(t, err)
assert.Nil(t, tagIds)
}
func TestGetTagIds_SingleId(t *testing.T) {
tagIds, err := TransactionTags.GetTagIds("1001")
assert.Nil(t, err)
assert.Equal(t, 1, len(tagIds))
assert.Equal(t, int64(1001), tagIds[0])
}
func TestGetTagIds_MultipleIds(t *testing.T) {
tagIds, err := TransactionTags.GetTagIds("1001,1002,1003")
assert.Nil(t, err)
assert.Equal(t, 3, len(tagIds))
assert.Equal(t, int64(1001), tagIds[0])
assert.Equal(t, int64(1002), tagIds[1])
assert.Equal(t, int64(1003), tagIds[2])
}
func TestGetTagIds_InvalidId(t *testing.T) {
tagIds, err := TransactionTags.GetTagIds("1001,invalid,1003")
assert.NotNil(t, err)
assert.Nil(t, tagIds)
}
func TestGetTransactionTagIds_EmptyMap(t *testing.T) {
allTransactionTagIds := make(map[int64][]int64)
actualTagIds := TransactionTags.GetTransactionTagIds(allTransactionTagIds)
assert.NotNil(t, actualTagIds)
assert.Equal(t, 0, len(actualTagIds))
}
func TestGetTransactionTagIds_MultipleTransactions(t *testing.T) {
allTransactionTagIds := map[int64][]int64{
1001: {2001, 2002},
1002: {2003},
1003: {2001, 2004},
}
actualTagIds := TransactionTags.GetTransactionTagIds(allTransactionTagIds)
assert.Equal(t, 5, len(actualTagIds))
assert.Contains(t, actualTagIds, int64(2001))
assert.Contains(t, actualTagIds, int64(2002))
assert.Contains(t, actualTagIds, int64(2003))
assert.Contains(t, actualTagIds, int64(2001))
assert.Contains(t, actualTagIds, int64(2004))
}
func TestGetTransactionTagIds_EmptyTransactionTagSlices(t *testing.T) {
allTransactionTagIds := map[int64][]int64{
1001: {},
1002: {},
}
actualTagIds := TransactionTags.GetTransactionTagIds(allTransactionTagIds)
assert.NotNil(t, actualTagIds)
assert.Equal(t, 0, len(actualTagIds))
}
func TestGetTransactionTagIds_MixedTransactionTagSlices(t *testing.T) {
allTransactionTagIds := map[int64][]int64{
1001: {2001, 2002},
1002: {},
1003: {2003},
}
actualTagIds := TransactionTags.GetTransactionTagIds(allTransactionTagIds)
assert.Equal(t, 3, len(actualTagIds))
assert.Contains(t, actualTagIds, int64(2001))
assert.Contains(t, actualTagIds, int64(2002))
assert.Contains(t, actualTagIds, int64(2003))
}
+76 -3
View File
@@ -1381,6 +1381,43 @@ func (s *TransactionService) DeleteAllTransactions(c core.Context, uid int64, de
})
}
// DeleteAllTransactionsOfAccount deletes all existed transactions of specific account from database
func (s *TransactionService) DeleteAllTransactionsOfAccount(c core.Context, uid int64, accountId int64, pageCount int32) error {
if uid <= 0 {
return errs.ErrUserIdInvalid
}
if accountId <= 0 {
return errs.ErrAccountIdInvalid
}
transactions, err := s.GetAllSpecifiedTransactions(c, uid, 0, 0, 0, nil, []int64{accountId}, nil, false, models.TRANSACTION_TAG_FILTER_HAS_ANY, "", "", pageCount, true)
if err != nil {
return err
}
if len(transactions) < 1 {
return nil
}
for i := 0; i < len(transactions); i++ {
transaction := transactions[i]
if transaction.Type == models.TRANSACTION_DB_TYPE_TRANSFER_IN {
err = s.DeleteTransaction(c, uid, transaction.RelatedId)
} else {
err = s.DeleteTransaction(c, uid, transaction.TransactionId)
}
if err != nil {
return err
}
}
return nil
}
// GetRelatedTransferTransaction returns the related transaction for transfer transaction
func (s *TransactionService) GetRelatedTransferTransaction(originalTransaction *models.Transaction) *models.Transaction {
var relatedType models.TransactionDbType
@@ -1422,7 +1459,7 @@ func (s *TransactionService) GetRelatedTransferTransaction(originalTransaction *
}
// GetAccountsTotalIncomeAndExpense returns the every accounts total income and expense amount by specific date range
func (s *TransactionService) GetAccountsTotalIncomeAndExpense(c core.Context, uid int64, startUnixTime int64, endUnixTime int64, utcOffset int16, useTransactionTimezone bool) (map[int64]int64, map[int64]int64, error) {
func (s *TransactionService) GetAccountsTotalIncomeAndExpense(c core.Context, uid int64, startUnixTime int64, endUnixTime int64, excludeAccountIds []int64, excludeCategoryIds []int64, utcOffset int16, useTransactionTimezone bool) (map[int64]int64, map[int64]int64, error) {
if uid <= 0 {
return nil, nil, errs.ErrUserIdInvalid
}
@@ -1437,13 +1474,49 @@ func (s *TransactionService) GetAccountsTotalIncomeAndExpense(c core.Context, ui
startTransactionTime := utils.GetMinTransactionTimeFromUnixTime(startUnixTime)
endTransactionTime := utils.GetMaxTransactionTimeFromUnixTime(endUnixTime)
condition := "uid=? AND deleted=? AND (type=? OR type=?) AND transaction_time>=? AND transaction_time<=?"
conditionParams := make([]any, 0, 4)
condition := "uid=? AND deleted=? AND (type=? OR type=?)"
conditionParams := make([]any, 0, 4+len(excludeAccountIds)+len(excludeCategoryIds))
conditionParams = append(conditionParams, uid)
conditionParams = append(conditionParams, false)
conditionParams = append(conditionParams, models.TRANSACTION_DB_TYPE_INCOME)
conditionParams = append(conditionParams, models.TRANSACTION_DB_TYPE_EXPENSE)
if len(excludeAccountIds) > 0 {
var accountIdsCondition strings.Builder
accountIdConditionParams := make([]any, 0, len(excludeAccountIds))
for i := 0; i < len(excludeAccountIds); i++ {
if i > 0 {
accountIdsCondition.WriteString(",")
}
accountIdsCondition.WriteString("?")
accountIdConditionParams = append(accountIdConditionParams, excludeAccountIds[i])
}
condition = condition + " AND account_id NOT IN (" + accountIdsCondition.String() + ")"
conditionParams = append(conditionParams, accountIdConditionParams...)
}
if len(excludeCategoryIds) > 0 {
var categoryIdsCondition strings.Builder
categoryIdConditionParams := make([]any, 0, len(excludeCategoryIds))
for i := 0; i < len(excludeCategoryIds); i++ {
if i > 0 {
categoryIdsCondition.WriteString(",")
}
categoryIdsCondition.WriteString("?")
categoryIdConditionParams = append(categoryIdConditionParams, excludeCategoryIds[i])
}
condition = condition + " AND category_id NOT IN (" + categoryIdsCondition.String() + ")"
conditionParams = append(conditionParams, categoryIdConditionParams...)
}
condition = condition + " AND transaction_time>=? AND transaction_time<=?"
minTransactionTime := startTransactionTime
maxTransactionTime := endTransactionTime
var allTransactions []*models.Transaction
+2 -2
View File
@@ -293,11 +293,11 @@ func (s *UserService) UpdateUser(c core.Context, user *models.User, modifyUserLa
updateCols = append(updateCols, "fiscal_year_start")
}
if core.CALENDAR_DISPLAY_TYPE_DEFAULT <= user.CalendarDisplayType && user.CalendarDisplayType <= core.CALENDAR_DISPLAY_TYPE_BUDDHIST {
if core.CALENDAR_DISPLAY_TYPE_DEFAULT <= user.CalendarDisplayType && user.CalendarDisplayType <= core.CALENDAR_DISPLAY_TYPE_GREGORAIN_WITH_PERSIAN {
updateCols = append(updateCols, "calendar_display_type")
}
if core.DATE_DISPLAY_TYPE_DEFAULT <= user.DateDisplayType && user.DateDisplayType <= core.DATE_DISPLAY_TYPE_BUDDHIST {
if core.DATE_DISPLAY_TYPE_DEFAULT <= user.DateDisplayType && user.DateDisplayType <= core.DATE_DISPLAY_TYPE_PERSIAN {
updateCols = append(updateCols, "date_display_type")
}
+129 -18
View File
@@ -66,6 +66,14 @@ const (
WebDAVStorageType string = "webdav"
)
const (
OpenAILLMProvider string = "openai"
OpenAICompatibleLLMProvider string = "openai_compatible"
OpenRouterLLMProvider string = "openrouter"
OllamaLLMProvider string = "ollama"
GoogleAILLMProvider string = "google_ai"
)
// Uuid generator types
const (
InternalUuidGeneratorType string = "internal"
@@ -140,6 +148,9 @@ const (
defaultWebDAVRequestTimeout uint32 = 10000 // 10 seconds
defaultAIRecognitionPictureMaxSize uint32 = 10485760 // 10MB
defaultLargeLanguageModelAPIRequestTimeout uint32 = 60000 // 60 seconds
defaultInMemoryDuplicateCheckerCleanupInterval uint32 = 60 // 1 minutes
defaultDuplicateSubmissionsInterval uint32 = 300 // 5 minutes
@@ -209,6 +220,25 @@ type WebDAVConfig struct {
SkipTLSVerify bool
}
// LLMConfig represents the Large Language Model setting config
type LLMConfig struct {
LLMProvider string
OpenAIAPIKey string
OpenAIModelID string
OpenAICompatibleBaseURL string
OpenAICompatibleAPIKey string
OpenAICompatibleModelID string
OpenRouterAPIKey string
OpenRouterModelID string
OllamaServerURL string
OllamaModelID string
GoogleAIAPIKey string
GoogleAIModelID string
LargeLanguageModelAPIRequestTimeout uint32
LargeLanguageModelAPIProxy string
LargeLanguageModelAPISkipTLSVerify bool
}
// TipConfig represents a tip setting config
type TipConfig struct {
Enabled bool
@@ -245,8 +275,9 @@ type Config struct {
StaticRootPath string
EnableGZip bool
EnableRequestLog bool
EnableGZip bool
EnableRequestLog bool
EnableRequestIdHeader bool
// MCP
EnableMCPServer bool
@@ -280,6 +311,13 @@ type Config struct {
MinIOConfig *MinIOConfig
WebDAVConfig *WebDAVConfig
// Large Language Model
TransactionFromAIImageRecognition bool
MaxAIRecognitionPictureFileSize uint32
// Large Language Model for Receipt Image Recognition
ReceiptImageRecognitionLLMConfig *LLMConfig
// Uuid
UuidGeneratorType string
UuidServerId uint8
@@ -299,7 +337,6 @@ type Config struct {
// Secret
SecretKeyNoSet bool
SecretKey string
EnableTwoFactor bool
TokenExpiredTime uint32
TokenExpiredTimeDuration time.Duration
TokenMinRefreshInterval uint32
@@ -311,20 +348,22 @@ type Config struct {
PasswordResetTokenExpiredTimeDuration time.Duration
MaxFailuresPerIpPerMinute uint32
MaxFailuresPerUserPerMinute uint32
EnableRequestIdHeader bool
// User
EnableUserRegister bool
EnableUserVerifyEmail bool
EnableUserForceVerifyEmail bool
// Auth
EnableTwoFactor bool
EnableUserForgetPassword bool
ForgetPasswordRequireVerifyEmail bool
EnableTransactionPictures bool
MaxTransactionPictureFileSize uint32
EnableScheduledTransaction bool
AvatarProvider core.UserAvatarProviderType
MaxAvatarFileSize uint32
DefaultFeatureRestrictions core.UserFeatureRestrictions
// User
EnableUserRegister bool
EnableUserVerifyEmail bool
EnableUserForceVerifyEmail bool
EnableTransactionPictures bool
MaxTransactionPictureFileSize uint32
EnableScheduledTransaction bool
AvatarProvider core.UserAvatarProviderType
MaxAvatarFileSize uint32
DefaultFeatureRestrictions core.UserFeatureRestrictions
// Data
EnableDataExport bool
@@ -426,6 +465,18 @@ func LoadConfiguration(configFilePath string) (*Config, error) {
return nil, err
}
err = loadLLMGlobalConfiguration(config, cfgFile, "llm")
if err != nil {
return nil, err
}
config.ReceiptImageRecognitionLLMConfig, err = loadLLMConfiguration(cfgFile, "llm_image_recognition")
if err != nil {
return nil, err
}
err = loadUuidConfiguration(config, cfgFile, "uuid")
if err != nil {
@@ -450,6 +501,12 @@ func LoadConfiguration(configFilePath string) (*Config, error) {
return nil, err
}
err = loadAuthConfiguration(config, cfgFile, "auth")
if err != nil {
return nil, err
}
err = loadUserConfiguration(config, cfgFile, "user")
if err != nil {
@@ -561,6 +618,7 @@ func loadServerConfiguration(config *Config, configFile *ini.File, sectionName s
config.EnableGZip = getConfigItemBoolValue(configFile, sectionName, "enable_gzip", false)
config.EnableRequestLog = getConfigItemBoolValue(configFile, sectionName, "log_request", false)
config.EnableRequestIdHeader = getConfigItemBoolValue(configFile, sectionName, "request_id_header", true)
return nil
}
@@ -750,6 +808,56 @@ func loadStorageConfiguration(config *Config, configFile *ini.File, sectionName
return nil
}
func loadLLMGlobalConfiguration(config *Config, configFile *ini.File, sectionName string) error {
config.TransactionFromAIImageRecognition = getConfigItemBoolValue(configFile, sectionName, "transaction_from_ai_image_recognition", false)
config.MaxAIRecognitionPictureFileSize = getConfigItemUint32Value(configFile, sectionName, "max_ai_recognition_picture_size", defaultAIRecognitionPictureMaxSize)
return nil
}
func loadLLMConfiguration(configFile *ini.File, sectionName string) (*LLMConfig, error) {
llmConfig := &LLMConfig{}
llmProvider := getConfigItemStringValue(configFile, sectionName, "llm_provider")
if llmProvider == "" {
llmConfig.LLMProvider = ""
} else if llmProvider == OpenAILLMProvider {
llmConfig.LLMProvider = OpenAILLMProvider
} else if llmProvider == OpenAICompatibleLLMProvider {
llmConfig.LLMProvider = OpenAICompatibleLLMProvider
} else if llmProvider == OpenRouterLLMProvider {
llmConfig.LLMProvider = OpenRouterLLMProvider
} else if llmProvider == OllamaLLMProvider {
llmConfig.LLMProvider = OllamaLLMProvider
} else if llmProvider == GoogleAILLMProvider {
llmConfig.LLMProvider = GoogleAILLMProvider
} else {
return nil, errs.ErrInvalidLLMProvider
}
llmConfig.OpenAIAPIKey = getConfigItemStringValue(configFile, sectionName, "openai_api_key")
llmConfig.OpenAIModelID = getConfigItemStringValue(configFile, sectionName, "openai_model_id")
llmConfig.OpenAICompatibleBaseURL = getConfigItemStringValue(configFile, sectionName, "openai_compatible_base_url")
llmConfig.OpenAICompatibleAPIKey = getConfigItemStringValue(configFile, sectionName, "openai_compatible_api_key")
llmConfig.OpenAICompatibleModelID = getConfigItemStringValue(configFile, sectionName, "openai_compatible_model_id")
llmConfig.OpenRouterAPIKey = getConfigItemStringValue(configFile, sectionName, "openrouter_api_key")
llmConfig.OpenRouterModelID = getConfigItemStringValue(configFile, sectionName, "openrouter_model_id")
llmConfig.OllamaServerURL = getConfigItemStringValue(configFile, sectionName, "ollama_server_url")
llmConfig.OllamaModelID = getConfigItemStringValue(configFile, sectionName, "ollama_model_id")
llmConfig.GoogleAIAPIKey = getConfigItemStringValue(configFile, sectionName, "google_ai_api_key")
llmConfig.GoogleAIModelID = getConfigItemStringValue(configFile, sectionName, "google_ai_model_id")
llmConfig.LargeLanguageModelAPIProxy = getConfigItemStringValue(configFile, sectionName, "proxy", "system")
llmConfig.LargeLanguageModelAPIRequestTimeout = getConfigItemUint32Value(configFile, sectionName, "request_timeout", defaultLargeLanguageModelAPIRequestTimeout)
llmConfig.LargeLanguageModelAPISkipTLSVerify = getConfigItemBoolValue(configFile, sectionName, "skip_tls_verify", false)
return llmConfig, nil
}
func loadUuidConfiguration(config *Config, configFile *ini.File, sectionName string) error {
if getConfigItemStringValue(configFile, sectionName, "generator_type") == InternalUuidGeneratorType {
config.UuidGeneratorType = InternalUuidGeneratorType
@@ -801,7 +909,6 @@ func loadCronConfiguration(config *Config, configFile *ini.File, sectionName str
func loadSecurityConfiguration(config *Config, configFile *ini.File, sectionName string) error {
config.SecretKeyNoSet = !getConfigItemIsSet(configFile, sectionName, "secret_key")
config.SecretKey = getConfigItemStringValue(configFile, sectionName, "secret_key", defaultSecretKey)
config.EnableTwoFactor = getConfigItemBoolValue(configFile, sectionName, "enable_two_factor", true)
config.TokenExpiredTime = getConfigItemUint32Value(configFile, sectionName, "token_expired_time", defaultTokenExpiredTime)
@@ -844,7 +951,13 @@ func loadSecurityConfiguration(config *Config, configFile *ini.File, sectionName
config.MaxFailuresPerIpPerMinute = getConfigItemUint32Value(configFile, sectionName, "max_failures_per_ip_per_minute", defaultMaxFailuresPerIpPerMinute)
config.MaxFailuresPerUserPerMinute = getConfigItemUint32Value(configFile, sectionName, "max_failures_per_user_per_minute", defaultMaxFailuresPerUserPerMinute)
config.EnableRequestIdHeader = getConfigItemBoolValue(configFile, sectionName, "request_id_header", true)
return nil
}
func loadAuthConfiguration(config *Config, configFile *ini.File, sectionName string) error {
config.EnableTwoFactor = getConfigItemBoolValue(configFile, sectionName, "enable_two_factor", true)
config.EnableUserForgetPassword = getConfigItemBoolValue(configFile, sectionName, "enable_forget_password", false)
config.ForgetPasswordRequireVerifyEmail = getConfigItemBoolValue(configFile, sectionName, "forget_password_require_email_verify", false)
return nil
}
@@ -853,8 +966,6 @@ func loadUserConfiguration(config *Config, configFile *ini.File, sectionName str
config.EnableUserRegister = getConfigItemBoolValue(configFile, sectionName, "enable_register", false)
config.EnableUserVerifyEmail = getConfigItemBoolValue(configFile, sectionName, "enable_email_verify", false)
config.EnableUserForceVerifyEmail = getConfigItemBoolValue(configFile, sectionName, "enable_force_email_verify", false)
config.EnableUserForgetPassword = getConfigItemBoolValue(configFile, sectionName, "enable_forget_password", false)
config.ForgetPasswordRequireVerifyEmail = getConfigItemBoolValue(configFile, sectionName, "forget_password_require_email_verify", false)
config.EnableTransactionPictures = getConfigItemBoolValue(configFile, sectionName, "enable_transaction_picture", false)
config.MaxTransactionPictureFileSize = getConfigItemUint32Value(configFile, sectionName, "max_transaction_picture_size", defaultTransactionPictureFileMaxSize)
config.EnableScheduledTransaction = getConfigItemBoolValue(configFile, sectionName, "enable_scheduled_transaction", false)
+6
View File
@@ -1,5 +1,7 @@
package settings
import "fmt"
// ConfigContainer contains the current setting config
type ConfigContainer struct {
current *Config
@@ -22,3 +24,7 @@ func SetCurrentConfig(config *Config) {
func (c *ConfigContainer) GetCurrentConfig() *Config {
return c.current
}
func GetUserAgent() string {
return fmt.Sprintf("ezBookkeeping/%s", Version)
}
+3 -2
View File
@@ -4,6 +4,7 @@ type KnownTemplate string
// Known templates
const (
TEMPLATE_VERIFY_EMAIL KnownTemplate = "email/verify_email"
TEMPLATE_PASSWORD_RESET KnownTemplate = "email/password_reset"
TEMPLATE_VERIFY_EMAIL KnownTemplate = "email/verify_email"
TEMPLATE_PASSWORD_RESET KnownTemplate = "email/password_reset"
SYSTEM_PROMPT_RECEIPT_IMAGE_RECOGNITION KnownTemplate = "prompt/receipt_image_recognition"
)
+3 -4
View File
@@ -5,7 +5,6 @@ import (
"net/http"
"reflect"
"github.com/gin-gonic/gin"
"github.com/go-playground/validator/v10"
"github.com/mayswind/ezbookkeeping/pkg/core"
@@ -14,7 +13,7 @@ import (
// PrintJsonSuccessResult writes success response in json format to current http context
func PrintJsonSuccessResult(c *core.WebContext, result any) {
c.JSON(http.StatusOK, gin.H{
c.JSON(http.StatusOK, core.O{
"success": true,
"result": result,
})
@@ -46,7 +45,7 @@ func PrintJsonErrorResult(c *core.WebContext, err *errs.Error) {
}
}
result := gin.H{
result := core.O{
"success": false,
"errorCode": err.Code(),
"errorMessage": errorMessage,
@@ -163,7 +162,7 @@ func WriteEventStreamJsonErrorResult(c *core.WebContext, originalErr *errs.Error
}
}
result := gin.H{
result := core.O{
"success": false,
"errorCode": originalErr.Code(),
"errorMessage": errorMessage,

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

+5 -5
View File
@@ -62,19 +62,19 @@ watch(currentNotificationContent, (newValue) => {
});
if (settingsStore.appSettings.theme === ThemeType.Light) {
theme.global.name.value = ThemeType.Light;
theme.change(ThemeType.Light);
} else if (settingsStore.appSettings.theme === ThemeType.Dark) {
theme.global.name.value = ThemeType.Dark;
theme.change(ThemeType.Dark);
} else {
theme.global.name.value = getSystemTheme();
theme.change(getSystemTheme());
}
window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', function (e) {
if (settingsStore.appSettings.theme === 'auto') {
if (e.matches) {
theme.global.name.value = ThemeType.Dark;
theme.change(ThemeType.Dark);
} else {
theme.global.name.value = ThemeType.Light;
theme.change(ThemeType.Light);
}
}
});
+3 -1
View File
@@ -23,7 +23,7 @@ import { useExchangeRatesStore } from '@/stores/exchangeRates.ts';
import { APPLICATION_LOGO_PATH } from '@/consts/asset.ts';
import { ThemeType } from '@/core/theme.ts';
import { isProduction } from '@/lib/version.ts';
import { getTheme, isEnableAnimate } from '@/lib/settings.ts';
import { getTheme, isEnableSwipeBack, isEnableAnimate } from '@/lib/settings.ts';
import { initMapProvider } from '@/lib/map/index.ts';
import { isUserLogined, isUserUnlocked } from '@/lib/userstate.ts';
import { setExpenseAndIncomeAmountColor } from '@/lib/ui/common.ts';
@@ -98,7 +98,9 @@ const f7params = ref<Framework7Parameters>({
browserHistory: !isiOSHomeScreenMode(),
browserHistoryInitialMatch: true,
browserHistoryAnimate: false,
iosSwipeBack: isEnableSwipeBack(),
iosSwipeBackAnimateShadow: false,
mdSwipeBack: isEnableSwipeBack(),
mdSwipeBackAnimateShadow: false
}
});
@@ -51,7 +51,13 @@ export interface CommonAccountBalanceTrendsChartProps {
}
export function useAccountBalanceTrendsChartBase(props: CommonAccountBalanceTrendsChartProps) {
const { formatUnixTimeToShortDate, formatUnixTimeToShortYear, formatUnixTimeToShortYearMonth, formatUnixTimeToYearQuarter, formatUnixTimeToFiscalYear } = useI18n();
const {
formatUnixTimeToShortDate,
formatUnixTimeToGregorianLikeShortYear,
formatUnixTimeToGregorianLikeShortYearMonth,
formatUnixTimeToGregorianLikeYearQuarter,
formatUnixTimeToGregorianLikeFiscalYear
} = useI18n();
const dataDateRange = computed<AccountBalanceUnixTimeAndBalanceRange | null>(() => {
if (!props.items || props.items.length < 1) {
@@ -63,9 +69,7 @@ export function useAccountBalanceTrendsChartBase(props: CommonAccountBalanceTren
let minUnixTimeClosingBalance = 0;
let maxUnixTimeClosingBalance = 0;
for (let i = 0; i < props.items.length; i++) {
const item = props.items[i];
for (const item of props.items) {
if (item.time < minUnixTime) {
minUnixTime = item.time;
minUnixTimeOpeningBalance = item.accountOpeningBalance;
@@ -114,8 +118,7 @@ export function useAccountBalanceTrendsChartBase(props: CommonAccountBalanceTren
const dayDataItemsMap: Record<number, TransactionReconciliationStatementResponseItem[]> = {};
for (let i = 0; i < props.items.length; i++) {
const dateItem = props.items[i];
for (const dateItem of props.items) {
let dateRangeMinUnixTime = 0;
if (props.dateAggregationType === ChartDateAggregationType.Year.type) {
@@ -143,20 +146,19 @@ export function useAccountBalanceTrendsChartBase(props: CommonAccountBalanceTren
let lastMedianBalance = lastClosingBalance;
let lastAverageBalance = lastClosingBalance;
for (let i = 0; i < allDateRanges.value.length; i++) {
const dateRange = allDateRanges.value[i];
for (const dateRange of allDateRanges.value) {
const dataItems = dayDataItemsMap[dateRange.minUnixTime];
let displayDate = '';
if (props.dateAggregationType === ChartDateAggregationType.Year.type) {
displayDate = formatUnixTimeToShortYear(dateRange.minUnixTime);
displayDate = formatUnixTimeToGregorianLikeShortYear(dateRange.minUnixTime);
} else if (props.dateAggregationType === ChartDateAggregationType.FiscalYear.type) {
displayDate = formatUnixTimeToFiscalYear(dateRange.minUnixTime);
displayDate = formatUnixTimeToGregorianLikeFiscalYear(dateRange.minUnixTime);
} else if (props.dateAggregationType === ChartDateAggregationType.Quarter.type) {
displayDate = formatUnixTimeToYearQuarter(dateRange.minUnixTime);
displayDate = formatUnixTimeToGregorianLikeYearQuarter(dateRange.minUnixTime);
} else if (props.dateAggregationType === ChartDateAggregationType.Month.type) {
displayDate = formatUnixTimeToShortYearMonth(dateRange.minUnixTime);
displayDate = formatUnixTimeToGregorianLikeShortYearMonth(dateRange.minUnixTime);
} else {
displayDate = formatUnixTimeToShortDate(dateRange.minUnixTime);
}
@@ -170,12 +172,12 @@ export function useAccountBalanceTrendsChartBase(props: CommonAccountBalanceTren
return data1.time - data2.time;
});
const openingBalance = dataItems[0].accountOpeningBalance;
const closingBalance = dataItems[dataItems.length - 1].accountClosingBalance;
const openingBalance = dataItems[0]!.accountOpeningBalance;
const closingBalance = dataItems[dataItems.length - 1]!.accountClosingBalance;
const minimumBalance = Math.min(...dataItems.map(item => item.accountClosingBalance));
const maximumBalance = Math.max(...dataItems.map(item => item.accountClosingBalance));
const medianBalance = dataItems[Math.floor(dataItems.length / 2)].accountClosingBalance;
const averageBalance = Math.floor(sumAmounts(dataItems.map(item => item.accountClosingBalance)) / dataItems.length);
const medianBalance = dataItems[Math.floor(dataItems.length / 2)]!.accountClosingBalance;
const averageBalance = Math.trunc(sumAmounts(dataItems.map(item => item.accountClosingBalance)) / dataItems.length);
if (props.account.isAsset) {
lastOpeningBalance = openingBalance;

Some files were not shown because too many files have changed in this diff Show More