Compare commits

..

100 Commits

Author SHA1 Message Date
MaysWind 06ef2220d6 fix the incorrect transaction type and amount when importing some Firefly III data 2025-07-13 16:00:53 +08:00
MaysWind 29d14bb5ef update latest supported currencies of Bank of Russia / International Monetary Fund exchange data source 2025-07-13 02:04:27 +08:00
MaysWind cd2b99a44c use the export data format since Firefly III version 6.2.0 as the format for importing Firefly III data 2025-07-13 01:53:17 +08:00
MaysWind 0413f8c0aa use the expense and revenue account names as category names if the transaction has not category when importing Firefly III transactions 2025-07-13 01:51:27 +08:00
MaysWind ca5c451d36 code refactor 2025-07-13 01:43:11 +08:00
MaysWind c19b87275d modify log content 2025-07-13 01:43:04 +08:00
MaysWind 7a374a509a modify file name 2025-07-12 16:08:35 +08:00
MaysWind 01aa2cf0a4 filter transaction description keywords in statistics & analysis page 2025-07-08 00:31:50 +08:00
MaysWind 5c9eb5dc5a modify style 2025-07-07 23:30:11 +08:00
MaysWind b05a53ffe3 reduce the number of skeleton rows when loading in transaction list page 2025-07-07 22:58:41 +08:00
MaysWind 0387551c43 change mcp token icon 2025-07-07 22:47:16 +08:00
MaysWind 773f808a35 update token last seen time when call mcp initialize api 2025-07-07 22:38:50 +08:00
MaysWind 07477eb5f8 hide generate mcp token when mcp is not enabled 2025-07-07 22:28:00 +08:00
MaysWind 5cb129311a feature restriction supports mcp 2025-07-07 01:21:09 +08:00
MaysWind 6215f489f2 code refactor 2025-07-07 01:20:55 +08:00
MaysWind 0140fc7622 add a special token type for MCP 2025-07-07 01:20:38 +08:00
MaysWind fbaf6086e3 update the Excelize version to the one actually used 2025-07-06 22:07:19 +08:00
MaysWind 5a1b649011 add transaction mcp handler 2025-07-06 22:03:26 +08:00
MaysWind 6da42686a9 add query accounts / transaction categories / transaction tags mcp handler 2025-07-06 20:02:42 +08:00
MaysWind 82b98eca95 code refactor 2025-07-06 20:02:09 +08:00
MaysWind a54275d307 fix missing text in description 2025-07-06 15:50:24 +08:00
MaysWind e1e61e8570 update git ignore file 2025-07-06 14:39:38 +08:00
MaysWind ebc7e7256a update description 2025-07-06 03:23:13 +08:00
MaysWind 93887ec2bb update README.md 2025-07-06 03:17:38 +08:00
MaysWind 8dce0f2d6a add mcp (Model Context Protocol) support 2025-07-06 03:02:19 +08:00
MaysWind 620ccf317f update README.md 2025-07-04 23:43:19 +08:00
MaysWind 7983f17e7f update documents 2025-07-03 00:08:18 +08:00
MaysWind b60c0b29f8 update README.md 2025-07-02 23:45:28 +08:00
MaysWind 5400a1424c do not check third party response when run tests in ci pipeline 2025-07-02 22:17:25 +08:00
MaysWind 3296d21f6a fix the bug that the date was not displayed correctly during daylight saving time (#163) 2025-07-02 01:40:20 +08:00
MaysWind 2e1a9362fc export transaction data based on the conditions on the transaction list page (#55) 2025-07-01 00:01:29 +08:00
MaysWind 53aa4ff390 code refactor 2025-06-30 22:58:17 +08:00
MaysWind 3c100b2543 code refactor 2025-06-30 22:42:18 +08:00
MaysWind b37cde5a8c user feature restriction supports application settings syncing 2025-06-30 22:10:54 +08:00
MaysWind e13efdc11f add sub category name in title 2025-06-30 21:53:36 +08:00
MaysWind 303f599f7d only show version dialog when frontend and backend version are not the same 2025-06-30 21:49:24 +08:00
MaysWind a68c45a923 fix typo 2025-06-30 21:42:59 +08:00
MaysWind 96b7c69283 add refresh browser cache when client version not match server version 2025-06-30 00:39:28 +08:00
MaysWind 801c0f8572 total amount on the account list page supports excluding specified accounts (#161) 2025-06-29 22:27:34 +08:00
MaysWind 90e862fbb1 sync application settings 2025-06-29 20:25:21 +08:00
MaysWind 1eb997d2c0 code refactor 2025-06-28 20:23:44 +08:00
MaysWind 1d314b1b09 fix the bug that statistical analysis still shows the account balance in the desktop version when the account balance is set to hide 2025-06-28 18:10:35 +08:00
MaysWind a077cccc2e upgrade golang to 1.24.4, node.js to 22.16.0, alpine base image to 3.22.0 2025-06-25 23:20:13 +08:00
MaysWind 6fb7e63e88 update description 2025-06-25 23:19:34 +08:00
MaysWind 3621245212 save / load column mapping file for delimiter-separated values (dsv) file 2025-06-22 22:49:06 +08:00
MaysWind dfa573b49b code refactor 2025-06-22 22:27:45 +08:00
MaysWind a69db9d299 use the macro language tag to match the i18n file when the browser language tag cannot match any i18n files 2025-06-22 18:53:25 +08:00
MaysWind 481618037d change the text of the unset start and end time in scheduled transaction 2025-06-22 18:07:11 +08:00
MaysWind 57ead2937b add Portuguese (Brazil) localized display name in different languages 2025-06-22 18:07:00 +08:00
MaysWind e6d8cbcdd6 fix wrong localized item key and default setting in Portuguese (Brazil) 2025-06-22 18:06:31 +08:00
MaysWind a7554d884f modify language order 2025-06-22 17:54:19 +08:00
MaysWind 468a4b1bac update default localized setting 2025-06-22 17:16:48 +08:00
Gustavo Michels de Camargo c1e4cd4bf1 Adding Brazilian Portuguese translation to the frontend 2025-06-22 15:21:38 +08:00
Gustavo Michels de Camargo 4413f2c411 Adding Brazilian Portuguese translation to the backend 2025-06-22 15:21:38 +08:00
MaysWind b1349f57cd parse information to account owner data in mt940 file 2025-06-21 00:52:08 +08:00
MaysWind 4a6f7eb43c import transactions from mt940 file 2025-06-20 00:57:07 +08:00
MaysWind 8f0e6ba95a support two-digit years in the transaction date when importing QIF file 2025-06-20 00:56:57 +08:00
MaysWind e9c175d2af code refactor 2025-06-20 00:55:59 +08:00
MaysWind 5dc0e925c1 fill the first two digits for year based on the current year when importing a two-digit year 2025-06-19 22:39:27 +08:00
MaysWind 787eaad352 add comments 2025-06-18 23:39:24 +08:00
MaysWind 4bab8db7c0 code refactor 2025-06-18 23:27:37 +08:00
MaysWind b6e96586a5 update README.md 2025-06-18 00:59:08 +08:00
MaysWind 7127c5539a import transactions from camt.053 file 2025-06-18 00:53:37 +08:00
MaysWind fe7736a7f6 export statistics data to markdown file 2025-06-15 23:33:57 +08:00
MaysWind 29dcaaae47 code refactor 2025-06-15 23:11:31 +08:00
MaysWind 9090c5c223 set geo location data order when import transaction 2025-06-15 22:59:21 +08:00
MaysWind 4336d1ed1a export statistics & analysis data in desktop version 2025-06-15 21:50:31 +08:00
MaysWind 8edc3640f5 code refactor 2025-06-15 21:48:56 +08:00
MaysWind 39e81af782 code refactor 2025-06-15 20:42:15 +08:00
MaysWind cc16f57a44 use vue-tsc instead of tsc to check code 2025-06-11 00:20:26 +08:00
MaysWind e7e7caae3b check typescript code in vue sfc file 2025-06-10 08:39:29 +08:00
Sebastian Reategui e09c62cf8d add missing fiscal year start parameter for src/lib/datetime.ts:getFullMonthDateRange() 2025-06-10 08:38:51 +08:00
MaysWind 8d5fe8f0f1 modify style 2025-06-10 00:10:57 +08:00
MaysWind f9d8293fd2 code refactor 2025-06-09 23:53:36 +08:00
MaysWind 4111eb0838 code refactor 2025-06-09 23:50:30 +08:00
MaysWind cd37e2ab1d fix the data of last quarter not displayed when there is only one month in the last quarter in trend analysis 2025-06-09 00:45:17 +08:00
MaysWind 2c730b3e25 code refactor 2025-06-09 00:33:16 +08:00
MaysWind ee47ee91c3 fix incorrect data aggregated by fiscal year in trend analysis 2025-06-08 23:10:02 +08:00
MaysWind 5a47c74f83 code refactor 2025-06-08 23:09:47 +08:00
MaysWind 0c4b8f006a import robustness 2025-06-08 22:10:04 +08:00
MaysWind 0023454d9a set default fiscal year start date when user registers 2025-06-08 22:03:25 +08:00
MaysWind 45e6c56934 modify style 2025-06-08 21:44:09 +08:00
MaysWind 51eb8fa377 code refactor 2025-06-08 21:44:02 +08:00
MaysWind f905dcb3fd improve compatibility 2025-06-08 02:47:30 +08:00
MaysWind 583676314a add multilingual entries 2025-06-08 02:47:24 +08:00
MaysWind 8616183660 do unit test when building frontend files 2025-06-08 02:47:10 +08:00
MaysWind ce4bca8272 code refactor 2025-06-08 02:47:00 +08:00
MaysWind 8c71f03f6f upgrade third party dependencies 2025-06-08 00:35:33 +08:00
MaysWind c5c4ddecbe remove redundant code 2025-06-07 23:00:53 +08:00
MaysWind ceecd9d524 code refactor 2025-06-07 23:00:35 +08:00
MaysWind a5a526e554 remove unused code 2025-06-07 22:14:26 +08:00
MaysWind 88864fd4f0 code refactor 2025-06-07 22:13:51 +08:00
MaysWind 10f2b39203 code refactor 2025-06-07 22:13:34 +08:00
MaysWind 6e1899c6ad format code 2025-06-07 22:13:16 +08:00
Sebastian Reategui b94dc8eb83 Feature - Add support for a fiscal year period defined in user settings.
* Add "This fiscal year", "Last fiscal year" as date range options in Transaction Details to filter transactions to those periods
* Add fiscal year ranges to Statistics & Trend Analysis
* Add "fiscal year start date" to user profile settings, allowing the user to select any date of the calendar year as the start of the fiscal year
* Add "fiscal year format" to user profile settings, allowing the user to specify how financial year date labels should appear

Implementation notes:
* The default fiscal year start is January 1 and the default fiscal year display format is "FY 2025"
* Fiscal year start date (month number & day number) are stored together in db as a uint16, high byte & low byte respectively
* February 29 is disallowed as a fiscal year start date, since it is never used as a convention in any country
* Jest is added to the project as a dev dependency, for unit tests in frontend

Signed-off-by: Sebastian Reategui <seb.reategui@gmail.com>
2025-06-07 22:04:47 +08:00
MaysWind 70eea8ff33 show total balance of parent account in mobile version (#149) 2025-06-07 14:11:55 +08:00
MaysWind 881a9c122a downgrade excelize to 2.9.0 (because https://github.com/qax-os/excelize/issues/2132) 2025-06-07 13:11:30 +08:00
MaysWind 9e9cac0c2e upgrade third party dependencies 2025-06-02 18:51:39 +08:00
MaysWind 83b2a3645d upgrade third party dependencies 2025-06-02 16:58:08 +08:00
MaysWind ecfca1c742 bump version to 0.10.0 2025-06-02 16:45:12 +08:00
288 changed files with 22248 additions and 4025 deletions
+2
View File
@@ -54,6 +54,8 @@ jobs:
push: true
build-args: |
RELEASE_BUILD=1
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+3 -1
View File
@@ -53,6 +53,8 @@ jobs:
platforms: ${{ vars.BUILD_SNAPSHOT_PLATFORMS }}
push: true
build-args: |
SKIP_TESTS=${{ vars.SKIP_TESTS }}
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+2
View File
@@ -50,6 +50,8 @@ jobs:
push: true
build-args: |
RELEASE_BUILD=1
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+2
View File
@@ -48,6 +48,8 @@ jobs:
linux/arm/v6
push: true
build-args: |
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
@@ -27,4 +27,6 @@ jobs:
platforms: linux/amd64
push: false
build-args: |
BUILD_PIPELINE=1
CHECK_3RD_API=${{ vars.CHECK_3RD_API }}
SKIP_TESTS=${{ vars.SKIP_TESTS }}
+3
View File
@@ -144,3 +144,6 @@ dist/
# Visual Studio Code
.vscode/
*.code-workspace
# Roo Code
.roo/
+9 -3
View File
@@ -1,8 +1,12 @@
# Build backend binary file
FROM golang:1.24.2-alpine3.21 AS be-builder
FROM golang:1.24.4-alpine3.22 AS be-builder
ARG RELEASE_BUILD
ARG BUILD_PIPELINE
ARG CHECK_3RD_API
ARG SKIP_TESTS
ENV RELEASE_BUILD=$RELEASE_BUILD
ENV BUILD_PIPELINE=$BUILD_PIPELINE
ENV CHECK_3RD_API=$CHECK_3RD_API
ENV SKIP_TESTS=$SKIP_TESTS
WORKDIR /go/src/github.com/mayswind/ezbookkeeping
COPY . .
@@ -11,9 +15,11 @@ RUN apk add git gcc g++ libc-dev
RUN ./build.sh backend
# Build frontend files
FROM --platform=$BUILDPLATFORM node:22.15.0-alpine3.21 AS fe-builder
FROM --platform=$BUILDPLATFORM node:22.16.0-alpine3.22 AS fe-builder
ARG RELEASE_BUILD
ARG BUILD_PIPELINE
ENV RELEASE_BUILD=$RELEASE_BUILD
ENV BUILD_PIPELINE=$BUILD_PIPELINE
WORKDIR /go/src/github.com/mayswind/ezbookkeeping
COPY . .
RUN docker/frontend-build-pre-setup.sh
@@ -21,7 +27,7 @@ RUN apk add git
RUN ./build.sh frontend
# Package docker image
FROM alpine:3.21.3
FROM alpine:3.22.0
LABEL maintainer="MaysWind <i@mayswind.net>"
RUN addgroup -S -g 1000 ezbookkeeping && adduser -S -G ezbookkeeping -u 1000 ezbookkeeping
RUN apk --no-cache add tzdata
+77 -39
View File
@@ -6,39 +6,45 @@
[![Latest Release](https://img.shields.io/github/release/mayswind/ezbookkeeping.svg?style=flat)](https://github.com/mayswind/ezbookkeeping/releases)
## Introduction
ezBookkeeping is a lightweight self-hosted personal bookkeeping app with user-friendly interface for both desktop and mobile devices. It supports PWA, you can [add the app homepage to the home screen](https://raw.githubusercontent.com/wiki/mayswind/ezbookkeeping/img/mobile/add_to_home_screen.gif) of your mobile device and use it just like a native app. It's easily to be deployed and configured, you can just deploy it by a single command via Docker. It supports almost all platforms, including Windows, macOS, and Linux, and is compatible with x86, amd64 and ARM hardware architectures. It only requires very few system resources, and you can even run it on a Raspberry Pi device.
ezBookkeeping is a lightweight, self-hosted personal finance app with a sleek, user-friendly interface and powerful bookkeeping features. Built with simplicity and portability in mind, it's easy to deploy, easy to use, and requires minimal system resources — perfect for microservers, NAS devices, and even Raspberry Pi.
Online Demo: [https://ezbookkeeping-demo.mayswind.net](https://ezbookkeeping-demo.mayswind.net)
The app is fully cross-platform and device-friendly — you can use it seamlessly on **mobile, tablet, and desktop devices**. With support for PWA (Progressive Web Apps), you can even [add it to your mobile home screen](https://raw.githubusercontent.com/wiki/mayswind/ezbookkeeping/img/mobile/add_to_home_screen.gif) and use it like a native app.
Live Demo: [https://ezbookkeeping-demo.mayswind.net](https://ezbookkeeping-demo.mayswind.net)
## Features
1. Open Source & Self-Hosted
2. Lightweight & Fast
3. Easy Installation
* Support Docker
* Support multiple databases (SQLite, MySQL, PostgreSQL, etc.)
* Support multiple operation system & hardware architectures (Windows, macOS, Linux & x86, amd64, ARM)
4. User-Friendly Interface
* Native UI for both desktop and mobile devices
* Support PWA, providing near-native experience for mobile devices
* Dark theme
5. Powerful Bookkeeping Features
* Support two-level account
* Support two-level transaction categories and predefined categories
* Support transaction pictures
* Support geographic location tracking and map
* Support recurring transactions
* Search and filter transaction records
* Data visualization and statistical analysis
6. Localization Support
* Multi-language support
* Multi-currency support with automatic exchange rate updates from various financial institutions
* Multi-timezone support
* Customizable date, time, number and currency display formats
7. Security & Reliability
* Two-factor authentication (2FA)
* Login rate limiting
* Application lock (PIN code / WebAuthn)
8. Data Export & Import (CSV, OFX, QFX, QIF, IIF, GnuCash, FireFly III, Beancount, etc.)
- **Open Source & Self-Hosted**
- Built for privacy and control
- **Lightweight & Fast**
- Optimized for performance, runs smoothly even on low-resource environments
- **Easy Installation**
- Docker-ready
- Supports SQLite, MySQL, PostgreSQL
- Cross-platform (Windows, macOS, Linux)
- Works on x86, amd64, ARM architectures
- **User-Friendly Interface**
- UI optimized for both mobile and desktop
- PWA support for native-like mobile experience
- Dark mode
- **AI-Powered Features**
- Supports MCP (Model Context Protocol) for AI integration
- **Powerful Bookkeeping**
- Two-level accounts and categories
- Attach images to transactions
- Location tracking with maps
- Recurring transactions
- Advanced filtering, search, visualization, and analysis
- **Localization & Globalization**
- Multi-language and multi-currency support
- Automatic exchange rates
- Multi-timezone awareness
- Custom formats for dates, numbers, and currencies
- **Security**
- Two-factor authentication (2FA)
- Login rate limiting
- Application lock (PIN code / WebAuthn)
- **Data Import/Export**
- Supports CSV, OFX, QFX, QIF, IIF, Camt.053, MT940, GnuCash, Firefly III, Beancount, and more
## Screenshots
### Desktop Version
@@ -48,19 +54,19 @@ Online Demo: [https://ezbookkeeping-demo.mayswind.net](https://ezbookkeeping-dem
[![ezBookkeeping](https://raw.githubusercontent.com/wiki/mayswind/ezbookkeeping/img/mobile/en.png)](https://raw.githubusercontent.com/wiki/mayswind/ezbookkeeping/img/mobile/en.png)
## Installation
### Ship with docker
### Run with Docker
Visit [Docker Hub](https://hub.docker.com/r/mayswind/ezbookkeeping) to see all images and tags.
Latest Release:
**Latest Release:**
$ docker run -p8080:8080 mayswind/ezbookkeeping
Latest Daily Build:
**Latest Daily Build:**
$ docker run -p8080:8080 mayswind/ezbookkeeping:latest-snapshot
### Install from binary
Latest release: [https://github.com/mayswind/ezbookkeeping/releases](https://github.com/mayswind/ezbookkeeping/releases)
### Install from Binary
Download the latest release: [https://github.com/mayswind/ezbookkeeping/releases](https://github.com/mayswind/ezbookkeeping/releases)
**Linux / macOS**
@@ -70,9 +76,9 @@ Latest release: [https://github.com/mayswind/ezbookkeeping/releases](https://git
> .\ezbookkeeping.exe server run
ezBookkeeping will listen at port 8080 as default. Then you can visit `http://{YOUR_HOST_ADDRESS}:8080/` .
By default, ezBookkeeping listens on port 8080. You can then visit `http://{YOUR_HOST_ADDRESS}:8080/` .
### Build from source
### Build from Source
Make sure you have [Golang](https://golang.org/), [GCC](http://gcc.gnu.org/), [Node.js](https://nodejs.org/) and [NPM](https://www.npmjs.com/) installed. Then download the source code, and follow these steps:
**Linux / macOS**
@@ -87,13 +93,45 @@ All the files will be packaged in `ezbookkeeping.tar.gz`.
All the files will be packaged in `ezbookkeeping.zip`.
You can also build docker image, make sure you have [docker](https://www.docker.com/) installed, then follow these steps:
You can also build a Docker image. Make sure you have [Docker](https://www.docker.com/) installed, then follow these steps:
**Linux**
$ ./build.sh docker
## Documents
## Contributing
We welcome contributions of all kinds!
Found a bug? [Submit an issue](https://github.com/mayswind/ezbookkeeping/issues)
Want to contribute code? Feel free to fork and send a pull request.
Contributions of all kinds — bug reports, feature suggestions, documentation improvements, or code — are highly appreciated.
Check out our [Contributor Graph](https://github.com/mayswind/ezbookkeeping/graphs/contributors) to see the amazing people whove already helped.
## Translating
Help make ezBookkeeping accessible to users around the world! If you want to contribute a translation, please refer to our [translation guide](https://ezbookkeeping.mayswind.net/translating).
Currently available translations:
| Tag | Language | Contributors |
| --- | --- | --- |
| de | Deutsch | [@chrgm](https://github.com/chrgm) |
| en | English | / |
| es | Español | [@Miguelonlonlon](https://github.com/Miguelonlonlon) |
| it | Italiano | [@waron97](https://github.com/waron97) |
| ja | 日本語 | [@tkymmm](https://github.com/tkymmm) |
| pt-BR | Português (Brasil) | [@thecodergus](https://github.com/thecodergus) |
| ru | Русский | [@artegoser](https://github.com/artegoser) |
| uk | Українська | [@nktlitvinenko](https://github.com/nktlitvinenko) |
| vi | Tiếng Việt | [@f97](https://github.com/f97) |
| zh-Hans | 中文 (简体) | / |
| zh-Hant | 中文 (繁體) | / |
Don't see your language? Help us add it!
## Documentation
1. [English](http://ezbookkeeping.mayswind.net)
1. [中文 (简体)](http://ezbookkeeping.mayswind.net/zh_Hans)
+11
View File
@@ -191,6 +191,17 @@ goto :pre_parse_args
)
)
if "%NO_TEST%"=="0" (
echo Executing frontend unit testing...
call npm run test
if !errorlevel! neq 0 (
call :echo_red "Error: Failed to pass unit testing"
goto :end
)
)
endlocal
echo Building frontend files(%RELEASE_TYPE%)...
+11
View File
@@ -179,6 +179,17 @@ build_frontend() {
fi
fi
if [ "$NO_TEST" = "0" ]; then
echo "Executing frontend unit testing..."
npm run test
if [ "$?" != "0" ]; then
echo_red "Error: Failed to pass unit testing"
exit 1
fi
fi
echo "Building frontend files ($RELEASE_TYPE)..."
if [ "$RELEASE" = "0" ]; then
+8
View File
@@ -141,5 +141,13 @@ func updateAllDatabaseTablesStructure(c *core.CliContext) error {
log.BootInfof(c, "[database.updateAllDatabaseTablesStructure] user custom exchange rate table maintained successfully")
err = datastore.Container.UserDataStore.SyncStructs(new(models.UserApplicationCloudSetting))
if err != nil {
return err
}
log.BootInfof(c, "[database.updateAllDatabaseTablesStructure] user application cloud settings table maintained successfully")
return nil
}
+20 -1
View File
@@ -260,6 +260,12 @@ var UserData = &cli.Command{
Required: true,
Usage: "Specific user name",
},
&cli.StringFlag{
Name: "type",
Aliases: []string{"t"},
Required: false,
Usage: "Specific token type, supports \"normal\" and \"mcp\", default is \"normal\"",
},
},
},
{
@@ -702,7 +708,18 @@ func createNewUserToken(c *core.CliContext) error {
}
username := c.String("username")
token, tokenString, err := clis.UserData.CreateNewUserToken(c, username)
tokenType := c.String("type")
if tokenType == "" {
tokenType = "normal"
}
if tokenType != "normal" && tokenType != "mcp" {
log.CliErrorf(c, "[user_data.createNewUserToken] token type is invalid")
return nil
}
token, tokenString, err := clis.UserData.CreateNewUserToken(c, username, tokenType)
if err != nil {
log.CliErrorf(c, "[user_data.createNewUserToken] error occurs when creating user token")
@@ -895,10 +912,12 @@ func printUserInfo(user *models.User) {
fmt.Printf("[Language] %s\n", user.Language)
fmt.Printf("[DefaultCurrency] %s\n", user.DefaultCurrency)
fmt.Printf("[FirstDayOfWeek] %s (%d)\n", user.FirstDayOfWeek, user.FirstDayOfWeek)
fmt.Printf("[FiscalYearStart] %s (%d)\n", user.FiscalYearStart, user.FiscalYearStart)
fmt.Printf("[LongDateFormat] %s (%d)\n", user.LongDateFormat, user.LongDateFormat)
fmt.Printf("[ShortDateFormat] %s (%d)\n", user.ShortDateFormat, user.ShortDateFormat)
fmt.Printf("[LongTimeFormat] %s (%d)\n", user.LongTimeFormat, user.LongTimeFormat)
fmt.Printf("[ShortTimeFormat] %s (%d)\n", user.ShortTimeFormat, user.ShortTimeFormat)
fmt.Printf("[FiscalYearFormat] %s (%d)\n", user.FiscalYearFormat, user.FiscalYearFormat)
fmt.Printf("[DecimalSeparator] %s (%d)\n", user.DecimalSeparator, user.DecimalSeparator)
fmt.Printf("[DigitGroupingSymbol] %s (%d)\n", user.DigitGroupingSymbol, user.DigitGroupingSymbol)
fmt.Printf("[DigitGrouping] %s (%d)\n", user.DigitGrouping, user.DigitGrouping)
+78
View File
@@ -2,6 +2,7 @@ package cmd
import (
"fmt"
"net/http"
"path/filepath"
"time"
@@ -18,6 +19,7 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/cron"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/mcp"
"github.com/mayswind/ezbookkeeping/pkg/middlewares"
"github.com/mayswind/ezbookkeeping/pkg/requestid"
"github.com/mayswind/ezbookkeeping/pkg/settings"
@@ -63,6 +65,13 @@ func startWebServer(c *core.CliContext) error {
return err
}
err = mcp.InitializeMCPHandlers(config)
if err != nil {
log.BootErrorf(c, "[webserver.startWebServer] initializes mcp handlers failed, because %s", err.Error())
return err
}
err = cron.InitializeCronJobSchedulerContainer(c, config, true)
if err != nil {
@@ -98,6 +107,7 @@ func startWebServer(c *core.CliContext) error {
_ = v.RegisterValidation("validCurrency", validators.ValidCurrency)
_ = v.RegisterValidation("validHexRGBColor", validators.ValidHexRGBColor)
_ = v.RegisterValidation("validAmountFilter", validators.ValidAmountFilter)
_ = v.RegisterValidation("validFiscalYearStart", validators.ValidateFiscalYearStart)
}
router.NoRoute(bindApi(api.Default.ApiNotFound))
@@ -211,6 +221,27 @@ func startWebServer(c *core.CliContext) error {
qrCodeRoute.GET("/mobile_url.png", bindCachedImage(api.QrCodes.MobileUrlQrCodeHandler, qrCodeCacheStore))
}
if config.EnableMCPServer {
mcpRoute := router.Group("/mcp")
mcpRoute.Use(bindMiddleware(middlewares.RequestId(config)))
mcpRoute.Use(bindMiddleware(middlewares.RequestLog))
mcpRoute.Use(bindMiddleware(middlewares.MCPServerIpLimit(config)))
mcpRoute.Use(bindMiddleware(middlewares.JWTMCPAuthorization))
{
mcpRoute.POST("", bindJSONRPCApi(map[string]core.JSONRPCApiHandlerFunc{
"initialize": api.ModelContextProtocols.InitializeHandler,
"resources/list": api.ModelContextProtocols.ListResourcesHandler,
"resources/read": api.ModelContextProtocols.ReadResourceHandler,
"tools/list": api.ModelContextProtocols.ListToolsHandler,
"tools/call": api.ModelContextProtocols.CallToolHandler,
"ping": api.ModelContextProtocols.PingHandler,
}, map[string]int{
"notifications/initialized": http.StatusAccepted,
}))
mcpRoute.GET("", bindApi(api.Default.MethodNotAllowed))
}
}
apiRoute := router.Group("/api")
apiRoute.Use(bindMiddleware(middlewares.RequestId(config)))
@@ -258,6 +289,7 @@ func startWebServer(c *core.CliContext) error {
{
// Tokens
apiV1Route.GET("/tokens/list.json", bindApi(api.Tokens.TokenListHandler))
apiV1Route.POST("/tokens/generate/mcp.json", bindApi(api.Tokens.TokenGenerateMCPHandler))
apiV1Route.POST("/tokens/revoke.json", bindApi(api.Tokens.TokenRevokeHandler))
apiV1Route.POST("/tokens/revoke_all.json", bindApi(api.Tokens.TokenRevokeAllHandler))
apiV1Route.POST("/tokens/refresh.json", bindApiWithTokenUpdate(api.Tokens.TokenRefreshHandler, config))
@@ -275,6 +307,11 @@ func startWebServer(c *core.CliContext) error {
apiV1Route.POST("/users/verify_email/resend.json", bindApi(api.Users.UserSendVerifyEmailByLoginedUserHandler))
}
// Application Cloud Settings
apiV1Route.GET("/users/settings/cloud/get.json", bindApi(api.UserApplicationCloudSettings.ApplicationSettingsGetHandler))
apiV1Route.POST("/users/settings/cloud/update.json", bindApi(api.UserApplicationCloudSettings.ApplicationSettingsUpdateHandler))
apiV1Route.POST("/users/settings/cloud/disable.json", bindApi(api.UserApplicationCloudSettings.ApplicationSettingsDisableHandler))
// Two-Factor Authorization
if config.EnableTwoFactor {
apiV1Route.GET("/users/2fa/status.json", bindApi(api.TwoFactorAuthorizations.TwoFactorStatusHandler))
@@ -361,6 +398,9 @@ func startWebServer(c *core.CliContext) error {
apiV1Route.GET("/exchange_rates/latest.json", bindApi(api.ExchangeRates.LatestExchangeRateHandler))
apiV1Route.POST("/exchange_rates/user_custom/update.json", bindApi(api.ExchangeRates.UserCustomExchangeRateUpdateHandler))
apiV1Route.POST("/exchange_rates/user_custom/delete.json", bindApi(api.ExchangeRates.UserCustomExchangeRateDeleteHandler))
// System
apiV1Route.GET("/systems/version.json", bindApi(api.Systems.VersionHandler))
}
}
@@ -423,6 +463,44 @@ func bindApiWithTokenUpdate(fn core.ApiHandlerFunc, config *settings.Config) gin
}
}
func bindJSONRPCApi(fns map[string]core.JSONRPCApiHandlerFunc, skipMethods map[string]int) gin.HandlerFunc {
return func(ginCtx *gin.Context) {
c := core.WrapWebContext(ginCtx)
var jsonRPCRequest core.JSONRPCRequest
reqErr := c.ShouldBindBodyWithJSON(&jsonRPCRequest)
if reqErr != nil {
utils.PrintJSONRPCErrorResult(c, nil, errs.NewIncompleteOrIncorrectSubmissionError(reqErr))
return
}
if skipMethods != nil {
httpStatusCode, exists := skipMethods[jsonRPCRequest.Method]
if exists {
c.AbortWithStatus(httpStatusCode)
return
}
}
fn, exists := fns[jsonRPCRequest.Method]
if !exists {
utils.PrintJSONRPCErrorResult(c, &jsonRPCRequest, errs.ErrApiNotFound)
return
}
result, err := fn(c, &jsonRPCRequest)
if err != nil {
utils.PrintJSONRPCErrorResult(c, &jsonRPCRequest, err)
} else {
utils.PrintJSONRPCSuccessResult(c, &jsonRPCRequest, result)
}
}
}
func bindEventStreamApi(fn core.EventStreamApiHandlerFunc) gin.HandlerFunc {
return func(ginCtx *gin.Context) {
c := core.WrapWebContext(ginCtx)
+9
View File
@@ -37,6 +37,13 @@ enable_gzip = false
# Set to true to log each request and execution time
log_request = true
[mcp]
# Set to true to enable MCP (Model Context Protocol) server (via http / https web server) for AI/LLM access
enable_mcp = false
# MCP server allowed remote IPs, a comma-separated list of allowed remote IPs (asterisk * for any addresses, e.g. 192.168.1.* means any IPs in the 192.168.1.x subnet), leave blank to allow all remote IPs
mcp_allowed_remote_ips =
[database]
# Either "mysql", "postgres" or "sqlite3"
type = sqlite3
@@ -236,6 +243,8 @@ max_user_avatar_size = 1048576
# 9: Import Transactions
# 10: Export Transactions
# 11: Clear All Data
# 12: Sync Application Settings
# 13: MCP (Model Context Protocol) Access
default_feature_restrictions =
[data]
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=ezBookkeeping, a lightweight personal bookkeeping app hosted by yourself.
Description=ezBookkeeping, a lightweight, self-hosted personal finance app with a sleek, user-friendly interface and powerful bookkeeping features.
After=syslog.target
After=network.target
After=mariadb.service mysqld.service postgresql.service
+2 -1
View File
@@ -28,10 +28,11 @@ var (
func main() {
settings.Version = Version
settings.CommitHash = CommitHash
settings.BuildTime = BuildUnixTime
cmd := &cli.Command{
Name: "ezBookkeeping",
Usage: "A lightweight personal bookkeeping app hosted by yourself.",
Usage: "A lightweight, self-hosted personal finance app with a sleek, user-friendly interface and powerful bookkeeping features.",
Version: GetFullVersion(),
Commands: []*cli.Command{
cmd.WebServer,
+22 -17
View File
@@ -5,26 +5,26 @@ go 1.24
require (
github.com/boombuler/barcode v1.0.2
github.com/extrame/xls v0.0.2-0.20200426124601-4a6cf263071b
github.com/gin-contrib/cache v1.3.2
github.com/gin-contrib/cache v1.4.0
github.com/gin-contrib/gzip v1.2.3
github.com/gin-gonic/gin v1.10.0
github.com/go-co-op/gocron/v2 v2.16.1
github.com/gin-gonic/gin v1.10.1
github.com/go-co-op/gocron/v2 v2.16.2
github.com/go-playground/validator/v10 v10.26.0
github.com/go-sql-driver/mysql v1.9.2
github.com/golang-jwt/jwt/v5 v5.2.2
github.com/lib/pq v1.10.9
github.com/mattn/go-sqlite3 v1.14.28
github.com/minio/minio-go/v7 v7.0.91
github.com/minio/minio-go/v7 v7.0.92
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pquerna/otp v1.4.0
github.com/pquerna/otp v1.5.0
github.com/sirupsen/logrus v1.9.3
github.com/stretchr/testify v1.10.0
github.com/urfave/cli/v3 v3.2.0
github.com/urfave/cli/v3 v3.3.3
github.com/wk8/go-ordered-map/v2 v2.1.8
github.com/xuri/excelize/v2 v2.9.0
golang.org/x/crypto v0.37.0
golang.org/x/net v0.39.0
golang.org/x/text v0.24.0
golang.org/x/crypto v0.38.0
golang.org/x/net v0.40.0
golang.org/x/text v0.25.0
gopkg.in/ini.v1 v1.67.0
gopkg.in/mail.v2 v2.3.1
xorm.io/builder v0.3.13
@@ -34,10 +34,11 @@ require (
require (
filippo.io/edwards25519 v1.1.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/bradfitz/gomemcache v0.0.0-20230905024940-24af94b03874 // indirect
github.com/bradfitz/gomemcache v0.0.0-20250403215159-8d39553ac7cf // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/bytedance/sonic v1.13.2 // indirect
github.com/bytedance/sonic/loader v0.2.4 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d // indirect
github.com/chenzhuoyu/iasm v0.9.1 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
@@ -47,8 +48,8 @@ require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/extrame/goyymmdd v0.0.0-20210114090516-7cc815f00d1a // indirect
github.com/extrame/ole2 v0.0.0-20160812065207-d69429661ad7 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gin-contrib/sse v1.0.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.9 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
@@ -56,6 +57,7 @@ require (
github.com/golang/snappy v0.0.4 // indirect
github.com/gomodule/redigo v1.9.2 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/jonboulle/clockwork v0.5.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.0 // indirect
@@ -69,7 +71,8 @@ require (
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/richardlehane/mscfb v1.0.4 // indirect
github.com/richardlehane/msoleps v1.0.4 // indirect
@@ -79,14 +82,16 @@ require (
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/syndtr/goleveldb v1.0.0 // indirect
github.com/tealeg/xlsx v1.0.5 // indirect
github.com/tiendc/go-deepcopy v1.6.0 // indirect
github.com/tinylib/msgp v1.3.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d // indirect
github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7 // indirect
golang.org/x/arch v0.15.0 // indirect
github.com/xuri/efp v0.0.1 // indirect
github.com/xuri/nfp v0.0.1 // indirect
golang.org/x/arch v0.17.0 // indirect
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8 // indirect
golang.org/x/sys v0.32.0 // indirect
golang.org/x/sys v0.33.0 // indirect
google.golang.org/protobuf v1.36.6 // indirect
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
+43 -36
View File
@@ -6,8 +6,8 @@ github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xW
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/boombuler/barcode v1.0.2 h1:79yrbttoZrLGkL/oOI8hBrUKucwOL0oOjUgEguGMcJ4=
github.com/boombuler/barcode v1.0.2/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/bradfitz/gomemcache v0.0.0-20230905024940-24af94b03874 h1:N7oVaKyGp8bttX0bfZGmcGkjz7DLQXhAn3DNd3T0ous=
github.com/bradfitz/gomemcache v0.0.0-20230905024940-24af94b03874/go.mod h1:r5xuitiExdLAJ09PR7vBVENGvp4ZuTBeWTGtxuX3K+c=
github.com/bradfitz/gomemcache v0.0.0-20250403215159-8d39553ac7cf h1:TqhNAT4zKbTdLa62d2HDBFdvgSbIGB3eJE8HqhgiL9I=
github.com/bradfitz/gomemcache v0.0.0-20250403215159-8d39553ac7cf/go.mod h1:r5xuitiExdLAJ09PR7vBVENGvp4ZuTBeWTGtxuX3K+c=
github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
@@ -17,6 +17,8 @@ github.com/bytedance/sonic v1.13.2/go.mod h1:o68xyaF9u2gvVBuGHPlUVCy+ZfmNNO5ETf1
github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
github.com/bytedance/sonic/loader v0.2.4 h1:ZWCw4stuXUsn1/+zQDqeE7JKP+QO47tz7QCNan80NzY=
github.com/bytedance/sonic/loader v0.2.4/go.mod h1:N8A3vUdtUebEY2/VQC0MyhYeKUFosQU6FxH2JmUe6VI=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d/go.mod h1:8EPpVsBuRksnlj1mLy4AWzRNQYxauNi62uWcE3to6eA=
@@ -38,18 +40,18 @@ github.com/extrame/ole2 v0.0.0-20160812065207-d69429661ad7/go.mod h1:GPpMrAfHdb8
github.com/extrame/xls v0.0.2-0.20200426124601-4a6cf263071b h1:jqW/h4gcXYEB6kVf6iuxjU9ONWA0ugUB94TP9UNmgdg=
github.com/extrame/xls v0.0.2-0.20200426124601-4a6cf263071b/go.mod h1:iACcgahst7BboCpIMSpnFs4SKyU9ZjsvZBfNbUxZOJI=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/gin-contrib/cache v1.3.2 h1:MsMTuG4KMhD2SVq5ygSYRci3BYdb/Egvk8lLNIB53gM=
github.com/gin-contrib/cache v1.3.2/go.mod h1:lnZv6QsBGSiqyB3rbNO2uVMWDBcMiZtHqH3Jlk57vaE=
github.com/gabriel-vasile/mimetype v1.4.9 h1:5k+WDwEsD9eTLL8Tz3L0VnmVh9QxGjRmjBvAG7U/oYY=
github.com/gabriel-vasile/mimetype v1.4.9/go.mod h1:WnSQhFKJuBlRyLiKohA/2DtIlPFAbguNaG7QCHcyGok=
github.com/gin-contrib/cache v1.4.0 h1:d1FUqCE2+gJQKT0vJjr7jMn1htW9+cypk5oF7aoQcmE=
github.com/gin-contrib/cache v1.4.0/go.mod h1:6d0UAPedInkublPl/uJUB4bqwsEgJI1y5QGszhqnyxg=
github.com/gin-contrib/gzip v1.2.3 h1:dAhT722RuEG330ce2agAs75z7yB+NKvX/ZM1r8w0u2U=
github.com/gin-contrib/gzip v1.2.3/go.mod h1:ad72i4Bzmaypk8M762gNXa2wkxxjbz0icRNnuLJ9a/c=
github.com/gin-contrib/sse v1.0.0 h1:y3bT1mUWUxDpW4JLQg/HnTqV4rozuW4tC9eFKTxYI9E=
github.com/gin-contrib/sse v1.0.0/go.mod h1:zNuFdwarAygJBht0NTKiSi3jRf6RbqeILZ9Sp6Slhe0=
github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
github.com/go-co-op/gocron/v2 v2.16.1 h1:ux/5zxVRveCaCuTtNI3DiOk581KC1KpJbpJFYUEVYwo=
github.com/go-co-op/gocron/v2 v2.16.1/go.mod h1:opexeOFy5BplhsKdA7bzY9zeYih8I8/WNJ4arTIFPVc=
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.10.1 h1:T0ujvqyCSqRopADpgPgiTT63DUQVSfojyME59Ei63pQ=
github.com/gin-gonic/gin v1.10.1/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
github.com/go-co-op/gocron/v2 v2.16.2 h1:r08P663ikXiulLT9XaabkLypL/W9MoCIbqgQoAutyX4=
github.com/go-co-op/gocron/v2 v2.16.2/go.mod h1:4YTLGCCAH75A5RlQ6q+h+VacO7CgjkgP0EJ+BEOXRSI=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
@@ -74,6 +76,8 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E=
github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I=
github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
@@ -105,8 +109,8 @@ github.com/minio/crc64nvme v1.0.1 h1:DHQPrYPdqK7jQG/Ls5CTBZWeex/2FMS3G5XGkycuFrY
github.com/minio/crc64nvme v1.0.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
github.com/minio/minio-go/v7 v7.0.91 h1:tWLZnEfo3OZl5PoXQwcwTAPNNrjyWwOh6cbZitW5JQc=
github.com/minio/minio-go/v7 v7.0.91/go.mod h1:uvMUcGrpgeSAAI6+sD3818508nUyMULw94j2Nxku/Go=
github.com/minio/minio-go/v7 v7.0.92 h1:jpBFWyRS3p8P/9tsRc+NuvqoFi7qAmTCFPoRFmobbVw=
github.com/minio/minio-go/v7 v7.0.92/go.mod h1:vTIc8DNcnAZIhyFsk8EB90AbPjj3j68aWIEQCiPj7d0=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -119,12 +123,14 @@ github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c h1:dAMKvw0MlJT1GshSTtih8C2gDs04w8dReiOGXrGLNoY=
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pquerna/otp v1.4.0 h1:wZvl1TIVxKRThZIBiwOOHOGP/1+nZyWBil9Y2XNEDzg=
github.com/pquerna/otp v1.4.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
github.com/pquerna/otp v1.5.0 h1:NMMR+WrmaqXU4EzdGJEE1aUUI0AMRzsp96fFFWNPwxs=
github.com/pquerna/otp v1.5.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
github.com/richardlehane/mscfb v1.0.4 h1:WULscsljNPConisD5hR0+OyZjwK46Pfyr6mPu5ZawpM=
github.com/richardlehane/mscfb v1.0.4/go.mod h1:YzVpcZg9czvAuhk9T+a3avCpcFPMUWm7gK3DypaEsUk=
github.com/richardlehane/msoleps v1.0.1/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg=
@@ -142,52 +148,53 @@ github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVs
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=
github.com/tealeg/xlsx v1.0.5 h1:+f8oFmvY8Gw1iUXzPk+kz+4GpbDZPK1FhPiQRd+ypgE=
github.com/tealeg/xlsx v1.0.5/go.mod h1:btRS8dz54TDnvKNosuAqxrM1QgN1udgk9O34bDCnORM=
github.com/tiendc/go-deepcopy v1.6.0/go.mod h1:toXoeQoUqXOOS/X4sKuiAoSk6elIdqc0pN7MTgOOo2I=
github.com/tinylib/msgp v1.3.0 h1:ULuf7GPooDaIlbyvgAxBV/FI7ynli6LZ1/nVUNu+0ww=
github.com/tinylib/msgp v1.3.0/go.mod h1:ykjzy2wzgrlvpDCRc4LA8UXy6D8bzMSuAF3WD57Gok0=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/urfave/cli/v3 v3.2.0 h1:m8WIXY0U9LCuUl5r+0fqLWDhNYWt6qvlW+GcF4EoXf8=
github.com/urfave/cli/v3 v3.2.0/go.mod h1:FJSKtM/9AiiTOJL4fJ6TbMUkxBXn7GO9guZqoZtpYpo=
github.com/urfave/cli/v3 v3.3.3 h1:byCBaVdIXuLPIDm5CYZRVG6NvT7tv1ECqdU4YzlEa3I=
github.com/urfave/cli/v3 v3.3.3/go.mod h1:FJSKtM/9AiiTOJL4fJ6TbMUkxBXn7GO9guZqoZtpYpo=
github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc=
github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d h1:llb0neMWDQe87IzJLS4Ci7psK/lVsjIS2otl+1WyRyY=
github.com/xuri/efp v0.0.0-20240408161823-9ad904a10d6d/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI=
github.com/xuri/efp v0.0.1 h1:fws5Rv3myXyYni8uwj2qKjVaRP30PdjeYe2Y6FDsCL8=
github.com/xuri/efp v0.0.1/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI=
github.com/xuri/excelize/v2 v2.9.0 h1:1tgOaEq92IOEumR1/JfYS/eR0KHOCsRv/rYXXh6YJQE=
github.com/xuri/excelize/v2 v2.9.0/go.mod h1:uqey4QBZ9gdMeWApPLdhm9x+9o2lq4iVmjiLfBS5hdE=
github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7 h1:hPVCafDV85blFTabnqKgNhDCkJX25eik94Si9cTER4A=
github.com/xuri/nfp v0.0.0-20240318013403-ab9948c2c4a7/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ=
github.com/xuri/nfp v0.0.1 h1:MDamSGatIvp8uOmDP8FnmjuQpu90NzdJxo7242ANR9Q=
github.com/xuri/nfp v0.0.1/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.15.0 h1:QtOrQd0bTUnhNVNndMpLHNWrDmYzZ2KDqSrEymqInZw=
golang.org/x/arch v0.15.0/go.mod h1:JmwW7aLIoRUKgaTzhkiEFxvcEiQGyOg9BMonBJUS7EE=
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
golang.org/x/arch v0.17.0 h1:4O3dfLzd+lQewptAHqjewQZQDyEdejz3VwgeYwkZneU=
golang.org/x/arch v0.17.0/go.mod h1:bdwinDaKcfZUGpH09BB7ZmOfhalA8lQdzl62l8gGWsk=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8/go.mod h1:jj3sYF3dwk5D+ghuXyeI3r5MFf+NT2An6/9dOA95KSI=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc h1:2gGKlE2+asNV9m7xrywl36YYNnBG5ZQ0r/BOOxqPpmk=
+21
View File
@@ -0,0 +1,21 @@
import { type JestConfigWithTsJest, createDefaultEsmPreset } from 'ts-jest';
const presetConfig = createDefaultEsmPreset({
tsconfig: '<rootDir>/tsconfig.jest.json'
});
const config: JestConfigWithTsJest = {
...presetConfig,
clearMocks: true,
collectCoverage: false,
moduleNameMapper: {
"^@/(.*)$": "<rootDir>/src/$1"
},
testEnvironment: "node",
testMatch: [
"**/__tests__/**/*.[jt]s?(x)",
"!**/__tests__/*_gen.[jt]s?(x)"
]
};
export default config;
+4452 -1213
View File
File diff suppressed because it is too large Load Diff
+21 -14
View File
@@ -1,6 +1,6 @@
{
"name": "ezbookkeeping",
"version": "0.9.0",
"version": "0.10.0",
"private": true,
"repository": {
"type": "git",
@@ -15,7 +15,8 @@
"serve": "cross-env NODE_ENV=development vite",
"build": "cross-env NODE_ENV=production vite build",
"serve:dist": "vite preview",
"lint": "tsc --noEmit && eslint . --fix"
"lint": "vue-tsc --noEmit && eslint . --fix",
"test": "cross-env TS_NODE_PROJECT=\"./tsconfig.jest.json\" jest"
},
"dependencies": {
"@mdi/js": "^7.4.47",
@@ -32,38 +33,44 @@
"leaflet": "^1.9.4",
"line-awesome": "^1.3.0",
"moment": "^2.30.1",
"moment-timezone": "^0.5.48",
"moment-timezone": "^0.6.0",
"pinia": "^3.0.2",
"register-service-worker": "^1.7.2",
"skeleton-elements": "^4.0.1",
"swiper": "^10.2.0",
"ua-parser-js": "^1.0.39",
"vue": "^3.5.13",
"vue": "^3.5.16",
"vue-echarts": "^7.0.3",
"vue-i18n": "^11.1.3",
"vue-i18n": "^11.1.5",
"vue-router": "^4.5.1",
"vue3-perfect-scrollbar": "^2.0.0",
"vuedraggable": "^4.1.0",
"vuetify": "^3.8.2"
"vuetify": "^3.8.7"
},
"devDependencies": {
"@tsconfig/node22": "^22.0.1",
"@jest/globals": "^29.7.0",
"@tsconfig/node22": "^22.0.2",
"@types/cbor-js": "^0.1.1",
"@types/crypto-js": "^4.2.2",
"@types/git-rev-sync": "^2.0.2",
"@types/node": "^22.15.2",
"@types/jest": "^29.5.14",
"@types/node": "^22.15.29",
"@types/ua-parser-js": "^0.7.39",
"@vitejs/plugin-vue": "^5.2.3",
"@vitejs/plugin-vue": "^5.2.4",
"@vue/eslint-config-typescript": "^14.5.0",
"@vue/tsconfig": "^0.7.0",
"cross-env": "^7.0.3",
"eslint": "^9.25.1",
"eslint-plugin-vue": "^10.0.0",
"eslint": "^9.28.0",
"eslint-plugin-vue": "^10.1.0",
"git-rev-sync": "^3.0.2",
"postcss-preset-env": "^10.1.6",
"sass": "^1.87.0",
"jest": "^29.7.0",
"postcss-preset-env": "^10.2.0",
"sass": "^1.89.1",
"ts-jest": "^29.3.4",
"ts-node": "^10.9.2",
"typescript": "^5.8.3",
"vite": "^6.3.3",
"vite": "^6.3.5",
"vite-plugin-checker": "^0.9.3",
"vite-plugin-pwa": "^1.0.0",
"vite-plugin-vuetify": "^2.1.1",
"vue-tsc": "^2.2.10"
+38 -8
View File
@@ -19,6 +19,7 @@ type AuthorizationsApi struct {
ApiUsingDuplicateChecker
ApiWithUserInfo
users *services.UserService
userAppCloudSettings *services.UserApplicationCloudSettingsService
tokens *services.TokenService
twoFactorAuthorizations *services.TwoFactorAuthorizationService
}
@@ -44,6 +45,7 @@ var (
},
},
users: services.Users,
userAppCloudSettings: services.UserApplicationCloudSettings,
tokens: services.Tokens,
twoFactorAuthorizations: services.TwoFactorAuthorizations,
}
@@ -140,9 +142,18 @@ func (a *AuthorizationsApi) AuthorizeHandler(c *core.WebContext) (any, *errs.Err
c.SetTokenClaims(claims)
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, user.Uid)
var applicationCloudSettingSlice *models.ApplicationCloudSettingSlice = nil
if err != nil {
log.Warnf(c, "[authorizations.AuthorizeHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", user.Uid, err.Error())
} else if userApplicationCloudSettings != nil && len(userApplicationCloudSettings.Settings) > 0 {
applicationCloudSettingSlice = &userApplicationCloudSettings.Settings
}
log.Infof(c, "[authorizations.AuthorizeHandler] user \"uid:%d\" has logined, token type is %d, token will be expired at %d", user.Uid, claims.Type, claims.ExpiresAt)
authResp := a.getAuthResponse(c, token, twoFactorEnable, user)
authResp := a.getAuthResponse(c, token, twoFactorEnable, user, applicationCloudSettingSlice)
return authResp, nil
}
@@ -218,9 +229,18 @@ func (a *AuthorizationsApi) TwoFactorAuthorizeHandler(c *core.WebContext) (any,
c.SetTextualToken(token)
c.SetTokenClaims(claims)
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, user.Uid)
var applicationCloudSettingSlice *models.ApplicationCloudSettingSlice = nil
if err != nil {
log.Warnf(c, "[authorizations.TwoFactorAuthorizeHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", user.Uid, err.Error())
} else if userApplicationCloudSettings != nil && len(userApplicationCloudSettings.Settings) > 0 {
applicationCloudSettingSlice = &userApplicationCloudSettings.Settings
}
log.Infof(c, "[authorizations.TwoFactorAuthorizeHandler] user \"uid:%d\" has authorized two-factor via passcode, token will be expired at %d", user.Uid, claims.ExpiresAt)
authResp := a.getAuthResponse(c, token, false, user)
authResp := a.getAuthResponse(c, token, false, user, applicationCloudSettingSlice)
return authResp, nil
}
@@ -303,17 +323,27 @@ func (a *AuthorizationsApi) TwoFactorAuthorizeByRecoveryCodeHandler(c *core.WebC
c.SetTextualToken(token)
c.SetTokenClaims(claims)
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, user.Uid)
var applicationCloudSettingSlice *models.ApplicationCloudSettingSlice = nil
if err != nil {
log.Warnf(c, "[authorizations.TwoFactorAuthorizeByRecoveryCodeHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", user.Uid, err.Error())
} else if userApplicationCloudSettings != nil && len(userApplicationCloudSettings.Settings) > 0 {
applicationCloudSettingSlice = &userApplicationCloudSettings.Settings
}
log.Infof(c, "[authorizations.TwoFactorAuthorizeByRecoveryCodeHandler] user \"uid:%d\" has authorized two-factor via recovery code \"%s\", token will be expired at %d", user.Uid, credential.RecoveryCode, claims.ExpiresAt)
authResp := a.getAuthResponse(c, token, false, user)
authResp := a.getAuthResponse(c, token, false, user, applicationCloudSettingSlice)
return authResp, nil
}
func (a *AuthorizationsApi) getAuthResponse(c *core.WebContext, token string, need2FA bool, user *models.User) *models.AuthResponse {
func (a *AuthorizationsApi) getAuthResponse(c *core.WebContext, token string, need2FA bool, user *models.User, applicationCloudSettings *models.ApplicationCloudSettingSlice) *models.AuthResponse {
return &models.AuthResponse{
Token: token,
Need2FA: need2FA,
User: a.GetUserBasicInfo(user),
NotificationContent: a.GetAfterLoginNotificationContent(user.Language, c.GetClientLocale()),
Token: token,
Need2FA: need2FA,
User: a.GetUserBasicInfo(user),
ApplicationCloudSettings: applicationCloudSettings,
NotificationContent: a.GetAfterLoginNotificationContent(user.Language, c.GetClientLocale()),
}
}
+46 -1
View File
@@ -197,6 +197,14 @@ func (a *DataManagementsApi) getExportedFileContent(c *core.WebContext, fileType
return nil, "", errs.ErrDataExportNotAllowed
}
var exportTransactionDataReq models.ExportTransactionDataRequest
err := c.ShouldBindQuery(&exportTransactionDataReq)
if err != nil {
log.Warnf(c, "[data_managements.ExportDataHandler] parse request failed, because %s", err.Error())
return nil, "", errs.NewIncompleteOrIncorrectSubmissionError(err)
}
timezone := time.Local
utcOffset, err := c.GetClientTimezoneOffset()
@@ -253,7 +261,44 @@ func (a *DataManagementsApi) getExportedFileContent(c *core.WebContext, fileType
categoryMap := a.categories.GetCategoryMapByList(categories)
tagMap := a.tags.GetTagMapByList(tags)
allTransactions, err := a.transactions.GetAllTransactions(c, uid, pageCountForDataExport, true)
allAccountIds, err := a.accounts.GetAccountOrSubAccountIds(c, exportTransactionDataReq.AccountIds, uid)
if err != nil {
log.Warnf(c, "[data_managements.ExportDataHandler] get account error, because %s", err.Error())
return nil, "", errs.Or(err, errs.ErrOperationFailed)
}
allCategoryIds, err := a.categories.GetCategoryOrSubCategoryIds(c, exportTransactionDataReq.CategoryIds, uid)
if err != nil {
log.Warnf(c, "[data_managements.ExportDataHandler] get transaction category error, because %s", err.Error())
return nil, "", errs.Or(err, errs.ErrOperationFailed)
}
var allTagIds []int64
noTags := exportTransactionDataReq.TagIds == "none"
if !noTags {
allTagIds, err = a.tags.GetTagIds(exportTransactionDataReq.TagIds)
if err != nil {
log.Warnf(c, "[data_managements.ExportDataHandler] get transaction tag ids error, because %s", err.Error())
return nil, "", errs.Or(err, errs.ErrOperationFailed)
}
}
maxTransactionTime := utils.GetMaxTransactionTimeFromUnixTime(time.Now().Unix())
minTransactionTime := int64(0)
if exportTransactionDataReq.MaxTime > 0 {
maxTransactionTime = utils.GetMaxTransactionTimeFromUnixTime(exportTransactionDataReq.MaxTime)
}
if exportTransactionDataReq.MinTime > 0 {
minTransactionTime = utils.GetMinTransactionTimeFromUnixTime(exportTransactionDataReq.MinTime)
}
allTransactions, err := a.transactions.GetAllSpecifiedTransactions(c, uid, maxTransactionTime, minTransactionTime, exportTransactionDataReq.Type, allCategoryIds, allAccountIds, allTagIds, noTags, exportTransactionDataReq.TagFilterType, exportTransactionDataReq.AmountFilter, exportTransactionDataReq.Keyword, pageCountForDataExport, true)
if err != nil {
log.Errorf(c, "[data_managements.ExportDataHandler] failed to all transactions user \"uid:%d\", because %s", uid, err.Error())
+267
View File
@@ -0,0 +1,267 @@
package api
import (
"encoding/json"
"github.com/gin-gonic/gin"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/mcp"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/services"
"github.com/mayswind/ezbookkeeping/pkg/settings"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const mcpServerName = "ezBookkeeping-mcp"
// ModelContextProtocolAPI represents model context protocol api
type ModelContextProtocolAPI struct {
ApiUsingConfig
transactions *services.TransactionService
transactionCategories *services.TransactionCategoryService
transactionTags *services.TransactionTagService
accounts *services.AccountService
users *services.UserService
tokens *services.TokenService
}
// Initialize a model context protocol api singleton instance
var (
ModelContextProtocols = &ModelContextProtocolAPI{
ApiUsingConfig: ApiUsingConfig{
container: settings.Container,
},
transactions: services.Transactions,
transactionCategories: services.TransactionCategories,
transactionTags: services.TransactionTags,
accounts: services.Accounts,
users: services.Users,
tokens: services.Tokens,
}
)
// InitializeHandler returns the initialize response for model context protocol
func (a *ModelContextProtocolAPI) InitializeHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
var initRequest mcp.MCPInitializeRequest
if jsonRPCRequest.Params != nil {
if err := json.Unmarshal(jsonRPCRequest.Params, &initRequest); err != nil {
return nil, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
} else {
return nil, errs.ErrIncompleteOrIncorrectSubmission
}
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[model_context_protocols.InitializeHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
tokenClaims := c.GetTokenClaims()
userTokenId, err := utils.StringToInt64(tokenClaims.UserTokenId)
if err != nil {
log.Warnf(c, "[model_context_protocols.InitializeHandler] parse user token id failed, because %s", err.Error())
} else {
tokenRecord := &models.TokenRecord{
Uid: tokenClaims.Uid,
UserTokenId: userTokenId,
CreatedUnixTime: tokenClaims.IssuedAt,
}
tokenId := a.tokens.GenerateTokenId(tokenRecord)
err = a.tokens.UpdateTokenLastSeen(c, tokenRecord)
if err != nil {
log.Warnf(c, "[model_context_protocols.InitializeHandler] failed to update last seen of token \"id:%s\" for user \"uid:%d\", because %s", tokenId, uid, err.Error())
}
}
protocolVersion := mcp.MCPProtocolVersion(initRequest.ProtocolVersion)
_, exists := mcp.SupportedMCPVersion[protocolVersion]
if !exists {
protocolVersion = mcp.LatestSupportedMCPVersion
}
initResp := mcp.MCPInitializeResponse{
ProtocolVersion: string(protocolVersion),
Capabilities: &mcp.MCPCapabilities{
Tools: &mcp.MCPToolCapabilities{
ListChanged: false,
},
},
ServerInfo: &mcp.MCPImplementation{
Name: mcpServerName,
Title: a.CurrentConfig().AppName,
Version: settings.Version,
},
}
return initResp, nil
}
// ListResourcesHandler returns the list of resources for model context protocol
func (a *ModelContextProtocolAPI) ListResourcesHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[model_context_protocols.ListResourcesHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
listResourcesResp := mcp.MCPListResourcesResponse{
Resources: make([]*mcp.MCPResource, 0),
}
return listResourcesResp, nil
}
// ReadResourceHandler returns the resource details for a specific resource in model context protocol
func (a *ModelContextProtocolAPI) ReadResourceHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
var readResourceReq mcp.MCPReadResourceRequest
if jsonRPCRequest.Params != nil {
if err := json.Unmarshal(jsonRPCRequest.Params, &readResourceReq); err != nil {
return nil, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
} else {
return nil, errs.ErrIncompleteOrIncorrectSubmission
}
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[model_context_protocols.ReadResourceHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
return nil, errs.ErrApiNotFound
}
// ListToolsHandler returns the list of tools for model context protocol
func (a *ModelContextProtocolAPI) ListToolsHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[model_context_protocols.ListToolsHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
mcpVersion := a.getMCPVersion(c)
toolsInfo := mcp.Container.GetMCPTools()
finalToolsInfos := make([]*mcp.MCPTool, len(toolsInfo))
for i := 0; i < len(toolsInfo); i++ {
finalToolsInfos[i] = &mcp.MCPTool{
Name: toolsInfo[i].Name,
InputSchema: toolsInfo[i].InputSchema,
Title: toolsInfo[i].Title,
Description: toolsInfo[i].Description,
}
if mcpVersion >= string(mcp.ToolResultStructuredContentMinVersion) {
finalToolsInfos[i].OutputSchema = toolsInfo[i].OutputSchema
}
}
listToolsResp := mcp.MCPListToolsResponse{
Tools: finalToolsInfos,
}
return listToolsResp, nil
}
// CallToolHandler returns the result of calling a specific tool for model context protocol
func (a *ModelContextProtocolAPI) CallToolHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[model_context_protocols.CallToolHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, errs.ErrNotPermittedToPerformThisAction
}
var callToolReq mcp.MCPCallToolRequest
if jsonRPCRequest.Params != nil {
if err := json.Unmarshal(jsonRPCRequest.Params, &callToolReq); err != nil {
return nil, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
} else {
return nil, errs.ErrIncompleteOrIncorrectSubmission
}
result, err := mcp.Container.HandleTool(c, &callToolReq, user, a.CurrentConfig(), a)
if err != nil {
return nil, errs.Or(err, errs.ErrOperationFailed)
}
return result, nil
}
// PingHandler return the ping response for model context protocol
func (a *ModelContextProtocolAPI) PingHandler(c *core.WebContext, jsonRPCRequest *core.JSONRPCRequest) (any, *errs.Error) {
return gin.H{}, nil
}
// GetTransactionService implements the MCPAvailableServices interface
func (a *ModelContextProtocolAPI) GetTransactionService() *services.TransactionService {
return a.transactions
}
// GetTransactionCategoryService implements the MCPAvailableServices interface
func (a *ModelContextProtocolAPI) GetTransactionCategoryService() *services.TransactionCategoryService {
return a.transactionCategories
}
// GetTransactionTagService implements the MCPAvailableServices interface
func (a *ModelContextProtocolAPI) GetTransactionTagService() *services.TransactionTagService {
return a.transactionTags
}
// GetAccountService implements the MCPAvailableServices interface
func (a *ModelContextProtocolAPI) GetAccountService() *services.AccountService {
return a.accounts
}
// GetUserService implements the MCPAvailableServices interface
func (a *ModelContextProtocolAPI) GetUserService() *services.UserService {
return a.users
}
// getMCPVersion returns the MCP protocol version from the request header
func (a *ModelContextProtocolAPI) getMCPVersion(c *core.WebContext) string {
return c.GetHeader(mcp.MCPProtocolVersionHeaderName)
}
+4
View File
@@ -43,6 +43,10 @@ func (a *ServerSettingsApi) ServerSettingsJavascriptHandler(c *core.WebContext)
a.appendBooleanSetting(builder, "e", config.EnableDataExport)
a.appendBooleanSetting(builder, "i", config.EnableDataImport)
if config.EnableMCPServer {
a.appendBooleanSetting(builder, "mcp", config.EnableMCPServer)
}
if config.LoginPageTips.Enabled {
a.appendMultiLanguageTipSetting(builder, "lpt", config.LoginPageTips)
}
+29
View File
@@ -0,0 +1,29 @@
package api
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/settings"
)
// SystemsApi represents system api
type SystemsApi struct{}
// Initialize a system api singleton instance
var (
Systems = &SystemsApi{}
)
// VersionHandler returns the server version and commit hash
func (a *SystemsApi) VersionHandler(c *core.WebContext) (any, *errs.Error) {
result := make(map[string]string)
result["version"] = settings.Version
result["commitHash"] = settings.CommitHash
if settings.BuildTime != "" {
result["buildTime"] = settings.BuildTime
}
return result, nil
}
+99 -26
View File
@@ -18,8 +18,9 @@ import (
type TokensApi struct {
ApiUsingConfig
ApiWithUserInfo
tokens *services.TokenService
users *services.UserService
tokens *services.TokenService
users *services.UserService
userAppCloudSettings *services.UserApplicationCloudSettingsService
}
// Initialize a token api singleton instance
@@ -36,15 +37,16 @@ var (
container: avatars.Container,
},
},
tokens: services.Tokens,
users: services.Users,
tokens: services.Tokens,
users: services.Users,
userAppCloudSettings: services.UserApplicationCloudSettings,
}
)
// TokenListHandler returns available token list of current user
func (a *TokensApi) TokenListHandler(c *core.WebContext) (any, *errs.Error) {
uid := c.GetCurrentUid()
tokens, err := a.tokens.GetAllUnexpiredNormalTokensByUid(c, uid)
tokens, err := a.tokens.GetAllUnexpiredNormalAndMCPTokensByUid(c, uid)
if err != nil {
log.Errorf(c, "[tokens.TokenListHandler] failed to get all tokens for user \"uid:%d\", because %s", uid, err.Error())
@@ -67,6 +69,10 @@ func (a *TokensApi) TokenListHandler(c *core.WebContext) (any, *errs.Error) {
tokenResp.IsCurrent = true
}
if token.TokenType == core.USER_TOKEN_TYPE_MCP && token.UserAgent != services.TokenUserAgentCreatedViaCli {
tokenResp.UserAgent = services.TokenUserAgentForMCP
}
tokenResps[i] = tokenResp
}
@@ -75,6 +81,53 @@ func (a *TokensApi) TokenListHandler(c *core.WebContext) (any, *errs.Error) {
return tokenResps, nil
}
// TokenGenerateMCPHandler generates a new MCP token for current user
func (a *TokensApi) TokenGenerateMCPHandler(c *core.WebContext) (any, *errs.Error) {
if !a.CurrentConfig().EnableMCPServer {
return nil, errs.ErrMCPServerNotEnabled
}
var generateMCPTokenReq models.TokenGenerateMCPRequest
err := c.ShouldBindJSON(&generateMCPTokenReq)
if err != nil {
log.Warnf(c, "[tokens.TokenGenerateMCPHandler] parse request failed, because %s", err.Error())
return nil, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[tokens.TokenGenerateMCPHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return false, errs.ErrNotPermittedToPerformThisAction
}
if !a.users.IsPasswordEqualsUserPassword(generateMCPTokenReq.Password, user) {
return nil, errs.ErrUserPasswordWrong
}
token, claims, err := a.tokens.CreateMCPToken(c, user)
if err != nil {
log.Errorf(c, "[tokens.TokenGenerateMCPHandler] failed to create mcp token for user \"uid:%d\", because %s", user.Uid, err.Error())
return nil, errs.Or(err, errs.ErrTokenGenerating)
}
log.Infof(c, "[tokens.TokenGenerateMCPHandler] user \"uid:%d\" has generated mcp token, new token will be expired at %d", user.Uid, claims.ExpiresAt)
generateMCPTokenResp := &models.TokenGenerateMCPResponse{
Token: token,
MCPUrl: a.CurrentConfig().RootUrl + "mcp",
}
return generateMCPTokenResp, nil
}
// TokenRevokeCurrentHandler revokes current token of current user
func (a *TokensApi) TokenRevokeCurrentHandler(c *core.WebContext) (any, *errs.Error) {
_, claims, err := a.tokens.ParseTokenByHeader(c)
@@ -100,11 +153,11 @@ func (a *TokensApi) TokenRevokeCurrentHandler(c *core.WebContext) (any, *errs.Er
err = a.tokens.DeleteToken(c, tokenRecord)
if err != nil {
log.Errorf(c, "[token.TokenRevokeCurrentHandler] failed to revoke token \"id:%s\" for user \"uid:%d\", because %s", tokenId, claims.Uid, err.Error())
log.Errorf(c, "[tokens.TokenRevokeCurrentHandler] failed to revoke token \"id:%s\" for user \"uid:%d\", because %s", tokenId, claims.Uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
log.Infof(c, "[token.TokenRevokeCurrentHandler] user \"uid:%d\" has revoked token \"id:%s\"", claims.Uid, tokenId)
log.Infof(c, "[tokens.TokenRevokeCurrentHandler] user \"uid:%d\" has revoked token \"id:%s\"", claims.Uid, tokenId)
return true, nil
}
@@ -122,7 +175,7 @@ func (a *TokensApi) TokenRevokeHandler(c *core.WebContext) (any, *errs.Error) {
if err != nil {
if !errs.IsCustomError(err) {
log.Errorf(c, "[token.TokenRevokeHandler] failed to parse token \"id:%s\", because %s", tokenRevokeReq.TokenId, err.Error())
log.Errorf(c, "[tokens.TokenRevokeHandler] failed to parse token \"id:%s\", because %s", tokenRevokeReq.TokenId, err.Error())
}
return nil, errs.Or(err, errs.ErrInvalidTokenId)
@@ -131,7 +184,7 @@ func (a *TokensApi) TokenRevokeHandler(c *core.WebContext) (any, *errs.Error) {
uid := c.GetCurrentUid()
if tokenRecord.Uid != uid {
log.Warnf(c, "[token.TokenRevokeHandler] token \"id:%s\" is not owned by user \"uid:%d\"", tokenRevokeReq.TokenId, uid)
log.Warnf(c, "[tokens.TokenRevokeHandler] token \"id:%s\" is not owned by user \"uid:%d\"", tokenRevokeReq.TokenId, uid)
return nil, errs.ErrInvalidTokenId
}
@@ -140,7 +193,7 @@ func (a *TokensApi) TokenRevokeHandler(c *core.WebContext) (any, *errs.Error) {
if err != nil {
if !errs.IsCustomError(err) {
log.Errorf(c, "[token.TokenRevokeHandler] failed to get user, because %s", err.Error())
log.Errorf(c, "[tokens.TokenRevokeHandler] failed to get user, because %s", err.Error())
}
return nil, errs.ErrUserNotFound
@@ -154,11 +207,11 @@ func (a *TokensApi) TokenRevokeHandler(c *core.WebContext) (any, *errs.Error) {
err = a.tokens.DeleteToken(c, tokenRecord)
if err != nil {
log.Errorf(c, "[token.TokenRevokeHandler] failed to revoke token \"id:%s\" for user \"uid:%d\", because %s", tokenRevokeReq.TokenId, uid, err.Error())
log.Errorf(c, "[tokens.TokenRevokeHandler] failed to revoke token \"id:%s\" for user \"uid:%d\", because %s", tokenRevokeReq.TokenId, uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
log.Infof(c, "[token.TokenRevokeHandler] user \"uid:%d\" has revoked token \"id:%s\"", uid, tokenRevokeReq.TokenId)
log.Infof(c, "[tokens.TokenRevokeHandler] user \"uid:%d\" has revoked token \"id:%s\"", uid, tokenRevokeReq.TokenId)
return true, nil
}
@@ -194,7 +247,7 @@ func (a *TokensApi) TokenRevokeAllHandler(c *core.WebContext) (any, *errs.Error)
if err != nil {
if !errs.IsCustomError(err) {
log.Errorf(c, "[token.TokenRevokeAllHandler] failed to get user, because %s", err.Error())
log.Errorf(c, "[tokens.TokenRevokeAllHandler] failed to get user, because %s", err.Error())
}
return nil, errs.ErrUserNotFound
@@ -207,11 +260,11 @@ func (a *TokensApi) TokenRevokeAllHandler(c *core.WebContext) (any, *errs.Error)
err = a.tokens.DeleteTokens(c, uid, tokens)
if err != nil {
log.Errorf(c, "[token.TokenRevokeAllHandler] failed to revoke all tokens for user \"uid:%d\", because %s", uid, err.Error())
log.Errorf(c, "[tokens.TokenRevokeAllHandler] failed to revoke all tokens for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
log.Infof(c, "[token.TokenRevokeAllHandler] user \"uid:%d\" has revoked all tokens", uid)
log.Infof(c, "[tokens.TokenRevokeAllHandler] user \"uid:%d\" has revoked all tokens", uid)
return true, nil
}
@@ -221,7 +274,7 @@ func (a *TokensApi) TokenRefreshHandler(c *core.WebContext) (any, *errs.Error) {
user, err := a.users.GetUserById(c, uid)
if err != nil {
log.Warnf(c, "[token.TokenRefreshHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
log.Warnf(c, "[tokens.TokenRefreshHandler] failed to get user \"uid:%d\" info, because %s", uid, err.Error())
return nil, errs.ErrUserNotFound
}
@@ -229,7 +282,7 @@ func (a *TokensApi) TokenRefreshHandler(c *core.WebContext) (any, *errs.Error) {
oldTokenClaims := c.GetTokenClaims()
if now-oldTokenClaims.IssuedAt < int64(a.CurrentConfig().TokenMinRefreshInterval) {
log.Infof(c, "[token.TokenRefreshHandler] token of user \"uid:%d\" does not need to be refreshed", uid)
log.Infof(c, "[tokens.TokenRefreshHandler] token of user \"uid:%d\" does not need to be refreshed", uid)
userTokenId, err := utils.StringToInt64(oldTokenClaims.UserTokenId)
@@ -247,13 +300,23 @@ func (a *TokensApi) TokenRefreshHandler(c *core.WebContext) (any, *errs.Error) {
err = a.tokens.UpdateTokenLastSeen(c, tokenRecord)
if err != nil {
log.Warnf(c, "[token.TokenRefreshHandler] failed to update last seen of token \"id:%s\" for user \"uid:%d\", because %s", tokenId, uid, err.Error())
log.Warnf(c, "[tokens.TokenRefreshHandler] failed to update last seen of token \"id:%s\" for user \"uid:%d\", because %s", tokenId, uid, err.Error())
}
}
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, user.Uid)
var applicationCloudSettingSlice *models.ApplicationCloudSettingSlice = nil
if err != nil {
log.Warnf(c, "[tokens.TokenRefreshHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", user.Uid, err.Error())
} else if userApplicationCloudSettings != nil && len(userApplicationCloudSettings.Settings) > 0 {
applicationCloudSettingSlice = &userApplicationCloudSettings.Settings
}
refreshResp := &models.TokenRefreshResponse{
User: a.GetUserBasicInfo(user),
NotificationContent: a.GetAfterOpenNotificationContent(user.Language, c.GetClientLocale()),
User: a.GetUserBasicInfo(user),
ApplicationCloudSettings: applicationCloudSettingSlice,
NotificationContent: a.GetAfterOpenNotificationContent(user.Language, c.GetClientLocale()),
}
return refreshResp, nil
@@ -262,7 +325,7 @@ func (a *TokensApi) TokenRefreshHandler(c *core.WebContext) (any, *errs.Error) {
token, claims, err := a.tokens.CreateToken(c, user)
if err != nil {
log.Errorf(c, "[token.TokenRefreshHandler] failed to create token for user \"uid:%d\", because %s", user.Uid, err.Error())
log.Errorf(c, "[tokens.TokenRefreshHandler] failed to create token for user \"uid:%d\", because %s", user.Uid, err.Error())
return nil, errs.Or(err, errs.ErrTokenGenerating)
}
@@ -276,13 +339,23 @@ func (a *TokensApi) TokenRefreshHandler(c *core.WebContext) (any, *errs.Error) {
c.SetTextualToken(token)
c.SetTokenClaims(claims)
log.Infof(c, "[token.TokenRefreshHandler] user \"uid:%d\" token refreshed, new token will be expired at %d", user.Uid, claims.ExpiresAt)
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, user.Uid)
var applicationCloudSettingSlice *models.ApplicationCloudSettingSlice = nil
if err != nil {
log.Warnf(c, "[tokens.TokenRefreshHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", user.Uid, err.Error())
} else if userApplicationCloudSettings != nil && len(userApplicationCloudSettings.Settings) > 0 {
applicationCloudSettingSlice = &userApplicationCloudSettings.Settings
}
log.Infof(c, "[tokens.TokenRefreshHandler] user \"uid:%d\" token refreshed, new token will be expired at %d", user.Uid, claims.ExpiresAt)
refreshResp := &models.TokenRefreshResponse{
NewToken: token,
OldTokenId: a.tokens.GenerateTokenId(oldTokenRecord),
User: a.GetUserBasicInfo(user),
NotificationContent: a.GetAfterOpenNotificationContent(user.Language, c.GetClientLocale()),
NewToken: token,
OldTokenId: a.tokens.GenerateTokenId(oldTokenRecord),
User: a.GetUserBasicInfo(user),
ApplicationCloudSettings: applicationCloudSettingSlice,
NotificationContent: a.GetAfterOpenNotificationContent(user.Language, c.GetClientLocale()),
}
return refreshResp, nil
+28 -152
View File
@@ -22,9 +22,6 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const maximumTagsCountOfTransaction = 10
const maximumPicturesCountOfTransaction = 10
// TransactionsApi represents transaction api
type TransactionsApi struct {
ApiUsingConfig
@@ -70,14 +67,14 @@ func (a *TransactionsApi) TransactionCountHandler(c *core.WebContext) (any, *err
uid := c.GetCurrentUid()
allAccountIds, err := a.getAccountOrSubAccountIds(c, transactionCountReq.AccountIds, uid)
allAccountIds, err := a.accounts.GetAccountOrSubAccountIds(c, transactionCountReq.AccountIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionCountHandler] get account error, because %s", err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
allCategoryIds, err := a.getCategoryOrSubCategoryIds(c, transactionCountReq.CategoryIds, uid)
allCategoryIds, err := a.transactionCategories.GetCategoryOrSubCategoryIds(c, transactionCountReq.CategoryIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionCountHandler] get transaction category error, because %s", err.Error())
@@ -88,7 +85,7 @@ func (a *TransactionsApi) TransactionCountHandler(c *core.WebContext) (any, *err
noTags := transactionCountReq.TagIds == "none"
if !noTags {
allTagIds, err = a.getTagIds(transactionCountReq.TagIds)
allTagIds, err = a.transactionTags.GetTagIds(transactionCountReq.TagIds)
if err != nil {
log.Warnf(c, "[transactions.TransactionCountHandler] get transaction tag ids error, because %s", err.Error())
@@ -138,14 +135,14 @@ func (a *TransactionsApi) TransactionListHandler(c *core.WebContext) (any, *errs
return nil, errs.ErrUserNotFound
}
allAccountIds, err := a.getAccountOrSubAccountIds(c, transactionListReq.AccountIds, uid)
allAccountIds, err := a.accounts.GetAccountOrSubAccountIds(c, transactionListReq.AccountIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionListHandler] get account error, because %s", err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
allCategoryIds, err := a.getCategoryOrSubCategoryIds(c, transactionListReq.CategoryIds, uid)
allCategoryIds, err := a.transactionCategories.GetCategoryOrSubCategoryIds(c, transactionListReq.CategoryIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionListHandler] get transaction category error, because %s", err.Error())
@@ -156,7 +153,7 @@ func (a *TransactionsApi) TransactionListHandler(c *core.WebContext) (any, *errs
noTags := transactionListReq.TagIds == "none"
if !noTags {
allTagIds, err = a.getTagIds(transactionListReq.TagIds)
allTagIds, err = a.transactionTags.GetTagIds(transactionListReq.TagIds)
if err != nil {
log.Warnf(c, "[transactions.TransactionListHandler] get transaction tag ids error, because %s", err.Error())
@@ -241,14 +238,14 @@ func (a *TransactionsApi) TransactionMonthListHandler(c *core.WebContext) (any,
return nil, errs.ErrUserNotFound
}
allAccountIds, err := a.getAccountOrSubAccountIds(c, transactionListReq.AccountIds, uid)
allAccountIds, err := a.accounts.GetAccountOrSubAccountIds(c, transactionListReq.AccountIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionMonthListHandler] get account error, because %s", err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
allCategoryIds, err := a.getCategoryOrSubCategoryIds(c, transactionListReq.CategoryIds, uid)
allCategoryIds, err := a.transactionCategories.GetCategoryOrSubCategoryIds(c, transactionListReq.CategoryIds, uid)
if err != nil {
log.Warnf(c, "[transactions.TransactionMonthListHandler] get transaction category error, because %s", err.Error())
@@ -259,7 +256,7 @@ func (a *TransactionsApi) TransactionMonthListHandler(c *core.WebContext) (any,
noTags := transactionListReq.TagIds == "none"
if !noTags {
allTagIds, err = a.getTagIds(transactionListReq.TagIds)
allTagIds, err = a.transactionTags.GetTagIds(transactionListReq.TagIds)
if err != nil {
log.Warnf(c, "[transactions.TransactionMonthListHandler] get transaction tag ids error, because %s", err.Error())
@@ -310,7 +307,7 @@ func (a *TransactionsApi) TransactionStatisticsHandler(c *core.WebContext) (any,
noTags := statisticReq.TagIds == "none"
if !noTags {
allTagIds, err = a.getTagIds(statisticReq.TagIds)
allTagIds, err = a.transactionTags.GetTagIds(statisticReq.TagIds)
if err != nil {
log.Warnf(c, "[transactions.TransactionStatisticsHandler] get transaction tag ids error, because %s", err.Error())
@@ -319,7 +316,7 @@ func (a *TransactionsApi) TransactionStatisticsHandler(c *core.WebContext) (any,
}
uid := c.GetCurrentUid()
totalAmounts, err := a.transactions.GetAccountsAndCategoriesTotalIncomeAndExpense(c, uid, statisticReq.StartTime, statisticReq.EndTime, allTagIds, noTags, statisticReq.TagFilterType, utcOffset, statisticReq.UseTransactionTimezone)
totalAmounts, err := a.transactions.GetAccountsAndCategoriesTotalIncomeAndExpense(c, uid, statisticReq.StartTime, statisticReq.EndTime, allTagIds, noTags, statisticReq.TagFilterType, statisticReq.Keyword, utcOffset, statisticReq.UseTransactionTimezone)
if err != nil {
log.Errorf(c, "[transactions.TransactionStatisticsHandler] failed to get accounts and categories total income and expense for user \"uid:%d\", because %s", uid, err.Error())
@@ -373,7 +370,7 @@ func (a *TransactionsApi) TransactionStatisticsTrendsHandler(c *core.WebContext)
noTags := statisticTrendsReq.TagIds == "none"
if !noTags {
allTagIds, err = a.getTagIds(statisticTrendsReq.TagIds)
allTagIds, err = a.transactionTags.GetTagIds(statisticTrendsReq.TagIds)
if err != nil {
log.Warnf(c, "[transactions.TransactionStatisticsTrendsHandler] get transaction tag ids error, because %s", err.Error())
@@ -382,7 +379,7 @@ func (a *TransactionsApi) TransactionStatisticsTrendsHandler(c *core.WebContext)
}
uid := c.GetCurrentUid()
allMonthlyTotalAmounts, err := a.transactions.GetAccountsAndCategoriesMonthlyIncomeAndExpense(c, uid, startYear, startMonth, endYear, endMonth, allTagIds, noTags, statisticTrendsReq.TagFilterType, utcOffset, statisticTrendsReq.UseTransactionTimezone)
allMonthlyTotalAmounts, err := a.transactions.GetAccountsAndCategoriesMonthlyIncomeAndExpense(c, uid, startYear, startMonth, endYear, endMonth, allTagIds, noTags, statisticTrendsReq.TagFilterType, statisticTrendsReq.Keyword, utcOffset, statisticTrendsReq.UseTransactionTimezone)
if err != nil {
log.Errorf(c, "[transactions.TransactionStatisticsTrendsHandler] failed to get accounts and categories total income and expense for user \"uid:%d\", because %s", uid, err.Error())
@@ -617,7 +614,7 @@ func (a *TransactionsApi) TransactionGetHandler(c *core.WebContext) (any, *errs.
}
if !transactionGetReq.TrimTag {
tagMap, err = a.transactionTags.GetTagsByTagIds(c, uid, utils.ToUniqueInt64Slice(a.getTransactionTagIds(allTransactionTagIds)))
tagMap, err = a.transactionTags.GetTagsByTagIds(c, uid, utils.ToUniqueInt64Slice(a.transactionTags.GetTransactionTagIds(allTransactionTagIds)))
if err != nil {
log.Errorf(c, "[transactions.TransactionGetHandler] failed to get transactions tags for user \"uid:%d\", because %s", uid, err.Error())
@@ -682,7 +679,7 @@ func (a *TransactionsApi) TransactionCreateHandler(c *core.WebContext) (any, *er
return nil, errs.ErrTransactionTagIdInvalid
}
if len(tagIds) > maximumTagsCountOfTransaction {
if len(tagIds) > models.MaximumTagsCountOfTransaction {
return nil, errs.ErrTransactionHasTooManyTags
}
@@ -693,7 +690,7 @@ func (a *TransactionsApi) TransactionCreateHandler(c *core.WebContext) (any, *er
return nil, errs.ErrTransactionPictureIdInvalid
}
if len(pictureIds) > maximumPicturesCountOfTransaction {
if len(pictureIds) > models.MaximumPicturesCountOfTransaction {
return nil, errs.ErrTransactionHasTooManyPictures
}
@@ -812,7 +809,7 @@ func (a *TransactionsApi) TransactionModifyHandler(c *core.WebContext) (any, *er
return nil, errs.ErrTransactionTagIdInvalid
}
if len(tagIds) > maximumTagsCountOfTransaction {
if len(tagIds) > models.MaximumTagsCountOfTransaction {
return nil, errs.ErrTransactionHasTooManyTags
}
@@ -823,7 +820,7 @@ func (a *TransactionsApi) TransactionModifyHandler(c *core.WebContext) (any, *er
return nil, errs.ErrTransactionPictureIdInvalid
}
if len(pictureIds) > maximumPicturesCountOfTransaction {
if len(pictureIds) > models.MaximumPicturesCountOfTransaction {
return nil, errs.ErrTransactionHasTooManyPictures
}
@@ -1219,6 +1216,13 @@ func (a *TransactionsApi) TransactionParseImportFileHandler(c *core.WebContext)
geoLocationSeparator = geoLocationSeparators[0]
}
geoLocationOrders := form.Value["geoOrder"]
geoLocationOrder := ""
if len(geoLocationOrders) > 0 {
geoLocationOrder = geoLocationOrders[0]
}
transactionTagSeparators := form.Value["tagSeparator"]
transactionTagSeparator := ""
@@ -1226,7 +1230,7 @@ func (a *TransactionsApi) TransactionParseImportFileHandler(c *core.WebContext)
transactionTagSeparator = transactionTagSeparators[0]
}
dataImporter, err = converters.CreateNewDelimiterSeparatedValuesDataImporter(fileType, fileEncoding, columnIndexMapping, transactionTypeNameMapping, hasHeaderLine, timeFormats[0], timezoneFormat, amountDecimalSeparator, amountDigitGroupingSymbol, geoLocationSeparator, transactionTagSeparator)
dataImporter, err = converters.CreateNewDelimiterSeparatedValuesDataImporter(fileType, fileEncoding, columnIndexMapping, transactionTypeNameMapping, hasHeaderLine, timeFormats[0], timezoneFormat, amountDecimalSeparator, amountDigitGroupingSymbol, geoLocationSeparator, geoLocationOrder, transactionTagSeparator)
} else {
dataImporter, err = converters.GetTransactionDataImporter(fileType)
}
@@ -1375,7 +1379,7 @@ func (a *TransactionsApi) TransactionImportHandler(c *core.WebContext) (any, *er
return nil, errs.ErrTransactionTagIdInvalid
}
if len(tagIds) > maximumTagsCountOfTransaction {
if len(tagIds) > models.MaximumTagsCountOfTransaction {
return nil, errs.ErrTransactionHasTooManyTags
}
@@ -1525,134 +1529,6 @@ func (a *TransactionsApi) filterTransactions(c *core.WebContext, uid int64, tran
return finalTransactions
}
func (a *TransactionsApi) getAccountOrSubAccountIds(c *core.WebContext, accountIds string, uid int64) ([]int64, error) {
if accountIds == "" || accountIds == "0" {
return nil, nil
}
requestAccountIds, err := utils.StringArrayToInt64Array(strings.Split(accountIds, ","))
if err != nil {
return nil, errs.Or(err, errs.ErrAccountIdInvalid)
}
var allAccountIds []int64
if len(requestAccountIds) > 0 {
allSubAccounts, err := a.accounts.GetSubAccountsByAccountIds(c, uid, requestAccountIds)
if err != nil {
return nil, err
}
accountIdsMap := make(map[int64]int32, len(requestAccountIds))
for i := 0; i < len(requestAccountIds); i++ {
accountIdsMap[requestAccountIds[i]] = 0
}
for i := 0; i < len(allSubAccounts); i++ {
subAccount := allSubAccounts[i]
if refCount, exists := accountIdsMap[subAccount.ParentAccountId]; exists {
accountIdsMap[subAccount.ParentAccountId] = refCount + 1
} else {
accountIdsMap[subAccount.ParentAccountId] = 1
}
if _, exists := accountIdsMap[subAccount.AccountId]; exists {
delete(accountIdsMap, subAccount.AccountId)
}
allAccountIds = append(allAccountIds, subAccount.AccountId)
}
for accountId, refCount := range accountIdsMap {
if refCount < 1 {
allAccountIds = append(allAccountIds, accountId)
}
}
}
return allAccountIds, nil
}
func (a *TransactionsApi) getCategoryOrSubCategoryIds(c *core.WebContext, categoryIds string, uid int64) ([]int64, error) {
if categoryIds == "" || categoryIds == "0" {
return nil, nil
}
requestCategoryIds, err := utils.StringArrayToInt64Array(strings.Split(categoryIds, ","))
if err != nil {
return nil, errs.Or(err, errs.ErrTransactionCategoryIdInvalid)
}
var allCategoryIds []int64
if len(requestCategoryIds) > 0 {
allSubCategories, err := a.transactionCategories.GetSubCategoriesByCategoryIds(c, uid, requestCategoryIds)
if err != nil {
return nil, err
}
categoryIdsMap := make(map[int64]int32, len(requestCategoryIds))
for i := 0; i < len(requestCategoryIds); i++ {
categoryIdsMap[requestCategoryIds[i]] = 0
}
for i := 0; i < len(allSubCategories); i++ {
subCategory := allSubCategories[i]
if refCount, exists := categoryIdsMap[subCategory.ParentCategoryId]; exists {
categoryIdsMap[subCategory.ParentCategoryId] = refCount + 1
} else {
categoryIdsMap[subCategory.ParentCategoryId] = 1
}
if _, exists := categoryIdsMap[subCategory.CategoryId]; exists {
delete(categoryIdsMap, subCategory.CategoryId)
}
allCategoryIds = append(allCategoryIds, subCategory.CategoryId)
}
for accountId, refCount := range categoryIdsMap {
if refCount < 1 {
allCategoryIds = append(allCategoryIds, accountId)
}
}
}
return allCategoryIds, nil
}
func (a *TransactionsApi) getTagIds(tagIds string) ([]int64, error) {
if tagIds == "" || tagIds == "0" {
return nil, nil
}
requestTagIds, err := utils.StringArrayToInt64Array(strings.Split(tagIds, ","))
if err != nil {
return nil, errs.Or(err, errs.ErrTransactionTagIdInvalid)
}
return requestTagIds, nil
}
func (a *TransactionsApi) getTransactionTagIds(allTransactionTagIds map[int64][]int64) []int64 {
allTagIds := make([]int64, 0, len(allTransactionTagIds))
for _, tagIds := range allTransactionTagIds {
allTagIds = append(allTagIds, tagIds...)
}
return allTagIds
}
func (a *TransactionsApi) getTransactionTagInfoResponses(tagIds []int64, allTransactionTags map[int64]*models.TransactionTag) []*models.TransactionTagInfoResponse {
allTags := make([]*models.TransactionTagInfoResponse, 0, len(tagIds))
@@ -1722,7 +1598,7 @@ func (a *TransactionsApi) getTransactionResponseListResult(c *core.WebContext, u
}
if !trimTag {
tagMap, err = a.transactionTags.GetTagsByTagIds(c, uid, utils.ToUniqueInt64Slice(a.getTransactionTagIds(allTransactionTagIds)))
tagMap, err = a.transactionTags.GetTagsByTagIds(c, uid, utils.ToUniqueInt64Slice(a.transactionTags.GetTransactionTagIds(allTransactionTagIds)))
if err != nil {
log.Errorf(c, "[transactions.getTransactionResponseListResult] failed to get transactions tags for user \"uid:%d\", because %s", uid, err.Error())
+220
View File
@@ -0,0 +1,220 @@
package api
import (
"encoding/json"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/services"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
// UserApplicationCloudSettingsApi represents user application cloud settings api
type UserApplicationCloudSettingsApi struct {
userAppCloudSettings *services.UserApplicationCloudSettingsService
users *services.UserService
}
// Initialize a user application cloud settings api singleton instance
var (
UserApplicationCloudSettings = &UserApplicationCloudSettingsApi{
userAppCloudSettings: services.UserApplicationCloudSettings,
users: services.Users,
}
)
// ApplicationSettingsGetHandler returns application cloud settings of current user
func (a *UserApplicationCloudSettingsApi) ApplicationSettingsGetHandler(c *core.WebContext) (any, *errs.Error) {
uid := c.GetCurrentUid()
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, uid)
if err != nil {
log.Errorf(c, "[user_app_cloud_settings.ApplicationSettingsGetHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", uid, err.Error())
return nil, errs.Or(err, errs.ErrOperationFailed)
}
if userApplicationCloudSettings == nil {
return false, nil
}
applicationCloudSettingSlice := userApplicationCloudSettings.Settings
if len(applicationCloudSettingSlice) < 1 {
return false, nil
}
return applicationCloudSettingSlice, nil
}
// ApplicationSettingsUpdateHandler updates user application cloud settings by request parameters for current user
func (a *UserApplicationCloudSettingsApi) ApplicationSettingsUpdateHandler(c *core.WebContext) (any, *errs.Error) {
var userAppCloudSettingUpdateReq models.UserApplicationCloudSettingsUpdateRequest
err := c.ShouldBindJSON(&userAppCloudSettingUpdateReq)
if err != nil {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] parse request failed, because %s", err.Error())
return false, errs.NewIncompleteOrIncorrectSubmissionError(err)
}
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
if !errs.IsCustomError(err) {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] failed to get user for user \"uid:%d\", because %s", uid, err.Error())
}
return false, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_SYNC_APPLICATION_SETTINGS) {
return false, errs.ErrNotPermittedToPerformThisAction
}
userApplicationCloudSettings, err := a.userAppCloudSettings.GetUserApplicationCloudSettingsByUid(c, uid)
if err != nil {
log.Errorf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] failed to get latest user application cloud settings for user \"uid:%d\", because %s", uid, err.Error())
return false, errs.Or(err, errs.ErrOperationFailed)
}
oldApplicationCloudSettingsMap := make(map[string]models.ApplicationCloudSetting)
if userApplicationCloudSettings != nil {
for _, setting := range userApplicationCloudSettings.Settings {
oldApplicationCloudSettingsMap[setting.SettingKey] = setting
}
}
// Check if the full update settings are the same as the existing settings
if userAppCloudSettingUpdateReq.FullUpdate {
if len(userAppCloudSettingUpdateReq.Settings) == len(oldApplicationCloudSettingsMap) {
needUpdate := false
for _, setting := range userAppCloudSettingUpdateReq.Settings {
oldSetting, exists := oldApplicationCloudSettingsMap[setting.SettingKey]
if !exists || oldSetting.SettingValue != setting.SettingValue {
needUpdate = true
break
}
}
if !needUpdate {
return false, errs.ErrNothingWillBeUpdated
}
}
} else { // Check if the partial update settings are the same as the existing settings or the settings to update are not set to sync
needUpdate := true
for _, setting := range userAppCloudSettingUpdateReq.Settings {
cloudSetting, exists := oldApplicationCloudSettingsMap[setting.SettingKey]
if !exists {
needUpdate = false
log.Infof(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" is not set to sync", setting.SettingKey)
} else if cloudSetting.SettingValue == setting.SettingValue {
needUpdate = false
log.Infof(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" value \"%s\" is not changed, no need to update", setting.SettingKey, setting.SettingValue)
}
}
if !needUpdate {
log.Infof(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] no user application cloud settings need to update for user \"uid:%d\"", uid)
return false, nil
}
}
newApplicationCloudSettingsMap := make(map[string]models.ApplicationCloudSetting)
var newApplicationCloudSettingSlice models.ApplicationCloudSettingSlice
if userAppCloudSettingUpdateReq.FullUpdate {
log.Infof(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user \"uid:%d\" application cloud settings force update, will overwrite all existing settings", uid)
} else {
if len(oldApplicationCloudSettingsMap) > 0 {
log.Infof(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user \"uid:%d\" application cloud settings exists, try to merge it with request settings", uid)
newApplicationCloudSettingsMap = oldApplicationCloudSettingsMap
}
}
for _, setting := range userAppCloudSettingUpdateReq.Settings {
newApplicationCloudSettingsMap[setting.SettingKey] = setting
}
for settingKey, setting := range newApplicationCloudSettingsMap {
settingType, exists := models.ALL_ALLOWED_CLOUD_SYNC_APP_SETTING_KEY_TYPES[settingKey]
if !exists {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" is not supported to sync", settingKey)
continue
}
if settingType == models.USER_APPLICATION_CLOUD_SETTING_TYPE_STRING {
// Do Nothing
} else if settingType == models.USER_APPLICATION_CLOUD_SETTING_TYPE_NUMBER {
_, err := utils.StringToFloat64(setting.SettingValue)
if err != nil {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" has invalid number value \"%s\"", settingKey, setting.SettingValue)
continue
}
} else if settingType == models.USER_APPLICATION_CLOUD_SETTING_TYPE_BOOLEAN {
if setting.SettingValue != "true" && setting.SettingValue != "false" {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" has invalid boolean value \"%s\"", settingKey, setting.SettingValue)
continue
}
} else if settingType == models.USER_APPLICATION_CLOUD_SETTING_TYPE_STRING_BOOLEAN_MAP {
var settingValueMap map[string]bool
err := json.Unmarshal([]byte(setting.SettingValue), &settingValueMap)
if err != nil {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" has invalid map value \"%s\", because %s", settingKey, setting.SettingValue, err.Error())
continue
}
} else {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] user application cloud setting key \"%s\" has unknown type \"%s\"", settingKey, settingType)
continue
}
newApplicationCloudSettingSlice = append(newApplicationCloudSettingSlice, setting)
}
err = a.userAppCloudSettings.UpdateUserApplicationCloudSettings(c, uid, newApplicationCloudSettingSlice)
if err != nil {
log.Errorf(c, "[user_app_cloud_settings.ApplicationSettingsUpdateHandler] failed to update user application cloud settings for user \"uid:%d\", because %s", uid, err.Error())
return false, errs.Or(err, errs.ErrOperationFailed)
}
return true, nil
}
// ApplicationSettingsDisableHandler disabled user application cloud settings by request parameters for current user
func (a *UserApplicationCloudSettingsApi) ApplicationSettingsDisableHandler(c *core.WebContext) (any, *errs.Error) {
uid := c.GetCurrentUid()
user, err := a.users.GetUserById(c, uid)
if err != nil {
if !errs.IsCustomError(err) {
log.Warnf(c, "[user_app_cloud_settings.ApplicationSettingsDisableHandler] failed to get user for user \"uid:%d\", because %s", uid, err.Error())
}
return false, errs.ErrUserNotFound
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_SYNC_APPLICATION_SETTINGS) {
return false, errs.ErrNotPermittedToPerformThisAction
}
err = a.userAppCloudSettings.ClearUserApplicationCloudSettings(c, uid)
if err != nil {
log.Errorf(c, "[user_app_cloud_settings.ApplicationSettingsDisableHandler] failed to clear user application cloud settings for user \"uid:%d\", because %s", uid, err.Error())
return false, errs.Or(err, errs.ErrOperationFailed)
}
return true, nil
}
+19
View File
@@ -78,6 +78,7 @@ func (a *UsersApi) UserRegisterHandler(c *core.WebContext) (any, *errs.Error) {
Language: userRegisterReq.Language,
DefaultCurrency: userRegisterReq.DefaultCurrency,
FirstDayOfWeek: userRegisterReq.FirstDayOfWeek,
FiscalYearStart: core.FISCAL_YEAR_START_DEFAULT,
TransactionEditScope: models.TRANSACTION_EDIT_SCOPE_ALL,
FeatureRestriction: a.CurrentConfig().DefaultFeatureRestrictions,
}
@@ -349,6 +350,15 @@ func (a *UsersApi) UserUpdateProfileHandler(c *core.WebContext) (any, *errs.Erro
userNew.FirstDayOfWeek = core.WEEKDAY_INVALID
}
if userUpdateReq.FiscalYearStart != nil && *userUpdateReq.FiscalYearStart != user.FiscalYearStart {
user.FiscalYearStart = *userUpdateReq.FiscalYearStart
userNew.FiscalYearStart = *userUpdateReq.FiscalYearStart
modifyProfileBasicInfo = true
anythingUpdate = true
} else {
userNew.FiscalYearStart = core.FISCAL_YEAR_START_INVALID
}
if userUpdateReq.LongDateFormat != nil && *userUpdateReq.LongDateFormat != user.LongDateFormat {
user.LongDateFormat = *userUpdateReq.LongDateFormat
userNew.LongDateFormat = *userUpdateReq.LongDateFormat
@@ -385,6 +395,15 @@ func (a *UsersApi) UserUpdateProfileHandler(c *core.WebContext) (any, *errs.Erro
userNew.ShortTimeFormat = core.SHORT_TIME_FORMAT_INVALID
}
if userUpdateReq.FiscalYearFormat != nil && *userUpdateReq.FiscalYearFormat != user.FiscalYearFormat {
user.FiscalYearFormat = *userUpdateReq.FiscalYearFormat
userNew.FiscalYearFormat = *userUpdateReq.FiscalYearFormat
modifyProfileBasicInfo = true
anythingUpdate = true
} else {
userNew.FiscalYearFormat = core.FISCAL_YEAR_FORMAT_INVALID
}
if userUpdateReq.DecimalSeparator != nil && *userUpdateReq.DecimalSeparator != user.DecimalSeparator {
user.DecimalSeparator = *userUpdateReq.DecimalSeparator
userNew.DecimalSeparator = *userUpdateReq.DecimalSeparator
+20 -3
View File
@@ -394,7 +394,7 @@ func (l *UserDataCli) ListUserTokens(c *core.CliContext, username string) ([]*mo
return nil, err
}
tokens, err := l.tokens.GetAllUnexpiredNormalTokensByUid(c, uid)
tokens, err := l.tokens.GetAllUnexpiredNormalAndMCPTokensByUid(c, uid)
if err != nil {
log.CliErrorf(c, "[user_data.ListUserTokens] failed to get tokens of user \"%s\", because %s", username, err.Error())
@@ -405,7 +405,7 @@ func (l *UserDataCli) ListUserTokens(c *core.CliContext, username string) ([]*mo
}
// CreateNewUserToken returns a new token for the specified user
func (l *UserDataCli) CreateNewUserToken(c *core.CliContext, username string) (*models.TokenRecord, string, error) {
func (l *UserDataCli) CreateNewUserToken(c *core.CliContext, username string, tokenType string) (*models.TokenRecord, string, error) {
if username == "" {
log.CliErrorf(c, "[user_data.CreateNewUserToken] user name is empty")
return nil, "", errs.ErrUsernameIsEmpty
@@ -418,7 +418,24 @@ func (l *UserDataCli) CreateNewUserToken(c *core.CliContext, username string) (*
return nil, "", err
}
token, tokenRecord, err := l.tokens.CreateTokenViaCli(c, user)
var token string
var tokenRecord *models.TokenRecord
if tokenType == "mcp" {
if !l.CurrentConfig().EnableMCPServer {
return nil, "", errs.ErrMCPServerNotEnabled
}
if user.FeatureRestriction.Contains(core.USER_FEATURE_RESTRICTION_TYPE_MCP_ACCESS) {
return nil, "", errs.ErrNotPermittedToPerformThisAction
}
token, tokenRecord, err = l.tokens.CreateMCPTokenViaCli(c, user)
} else if tokenType == "normal" {
token, tokenRecord, err = l.tokens.CreateTokenViaCli(c, user)
} else {
return nil, "", errs.ErrParameterInvalid
}
if err != nil {
log.CliErrorf(c, "[user_data.CreateNewUserToken] failed to create token for user \"%s\", because %s", username, err.Error())
@@ -61,13 +61,13 @@ func (c *alipayTransactionDataCsvFileImporter) ParseImportedData(ctx core.Contex
enc := simplifiedchinese.GB18030
reader := transform.NewReader(bytes.NewReader(data), enc.NewDecoder())
dataTable, err := c.createNewAlipayImportedDataTable(ctx, reader, c.fileHeaderLine, c.dataHeaderStartContent, c.dataBottomEndLineRune)
dataTable, err := c.createNewAlipayBasicDataTable(ctx, reader, c.fileHeaderLine, c.dataHeaderStartContent, c.dataBottomEndLineRune)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
commonDataTable := datatable.CreateNewImportedCommonDataTable(dataTable)
commonDataTable := datatable.CreateNewCommonDataTableFromBasicDataTable(dataTable)
if !commonDataTable.HasColumn(c.originalColumnNames.timeColumnName) ||
!commonDataTable.HasColumn(c.originalColumnNames.amountColumnName) ||
@@ -77,14 +77,14 @@ func (c *alipayTransactionDataCsvFileImporter) ParseImportedData(ctx core.Contex
return nil, nil, nil, nil, nil, nil, errs.ErrMissingRequiredFieldInHeaderRow
}
transactionRowParser := createAlipayTransactionDataRowParser(c.originalColumnNames)
transactionDataTable := datatable.CreateNewCommonTransactionDataTable(commonDataTable, alipayTransactionSupportedColumns, transactionRowParser)
transactionRowParser := createAlipayTransactionDataRowParser(c.originalColumnNames, dataTable.HeaderColumnNames())
transactionDataTable := datatable.CreateNewTransactionDataTableFromCommonDataTable(commonDataTable, alipayTransactionSupportedColumns, transactionRowParser)
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(alipayTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
func (c *alipayTransactionDataCsvFileImporter) createNewAlipayImportedDataTable(ctx core.Context, reader io.Reader, fileHeaderLine string, dataHeaderStartContent string, dataBottomEndLineRune rune) (datatable.ImportedDataTable, error) {
func (c *alipayTransactionDataCsvFileImporter) createNewAlipayBasicDataTable(ctx core.Context, reader io.Reader, fileHeaderLine string, dataHeaderStartContent string, dataBottomEndLineRune rune) (datatable.BasicDataTable, error) {
csvReader := csv.NewReader(reader)
csvReader.FieldsPerRecord = -1
@@ -100,7 +100,7 @@ func (c *alipayTransactionDataCsvFileImporter) createNewAlipayImportedDataTable(
}
if err != nil {
log.Errorf(ctx, "[alipay_transaction_csv_data_table.createNewAlipayImportedDataTable] cannot parse alipay csv data, because %s", err.Error())
log.Errorf(ctx, "[alipay_transaction_data_csv_file_importer.createNewAlipayBasicDataTable] cannot parse alipay csv data, because %s", err.Error())
return nil, errs.ErrInvalidCSVFile
}
@@ -111,7 +111,7 @@ func (c *alipayTransactionDataCsvFileImporter) createNewAlipayImportedDataTable(
hasFileHeader = true
continue
} else {
log.Warnf(ctx, "[alipay_transaction_csv_data_table.createNewAlipayImportedDataTable] read unexpected line before read file header, line content is %s", strings.Join(items, ","))
log.Warnf(ctx, "[alipay_transaction_data_csv_file_importer.createNewAlipayBasicDataTable] read unexpected line before read file header, line content is %s", strings.Join(items, ","))
continue
}
}
@@ -139,7 +139,7 @@ func (c *alipayTransactionDataCsvFileImporter) createNewAlipayImportedDataTable(
}
if len(allOriginalLines) > 0 && len(items) < len(allOriginalLines[0]) {
log.Errorf(ctx, "[alipay_transaction_csv_data_table.createNewAlipayImportedDataTable] cannot parse row \"index:%d\", because may missing some columns (column count %d in data row is less than header column count %d)", len(allOriginalLines), len(items), len(allOriginalLines[0]))
log.Errorf(ctx, "[alipay_transaction_data_csv_file_importer.createNewAlipayBasicDataTable] cannot parse row \"index:%d\", because may missing some columns (column count %d in data row is less than header column count %d)", len(allOriginalLines), len(items), len(allOriginalLines[0]))
return nil, errs.ErrFewerFieldsInDataRowThanInHeaderRow
}
@@ -152,11 +152,11 @@ func (c *alipayTransactionDataCsvFileImporter) createNewAlipayImportedDataTable(
}
if len(allOriginalLines) < 2 {
log.Errorf(ctx, "[alipay_transaction_csv_data_table.createNewAlipayImportedDataTable] cannot parse import data, because data table row count is less 1")
log.Errorf(ctx, "[alipay_transaction_data_csv_file_importer.createNewAlipayBasicDataTable] cannot parse import data, because data table row count is less 1")
return nil, errs.ErrNotFoundTransactionDataInFile
}
dataTable := csvdatatable.CreateNewCustomCsvImportedDataTable(allOriginalLines)
dataTable := csvdatatable.CreateNewCustomCsvBasicDataTable(allOriginalLines)
return dataTable, nil
}
@@ -26,11 +26,12 @@ const alipayTransactionDataProductNameRepaymentText = "还款"
// alipayTransactionDataRowParser defines the structure of alipay transaction data row parser
type alipayTransactionDataRowParser struct {
columns alipayTransactionColumnNames
columns alipayTransactionColumnNames
existedOriginalDataColumns map[string]bool
}
// Parse returns the converted transaction data row
func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataTable *datatable.CommonTransactionDataTable, dataRow datatable.CommonDataRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataRow datatable.CommonDataTableRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
if dataRow.GetData(p.columns.typeColumnName) != alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] &&
dataRow.GetData(p.columns.typeColumnName) != alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE] &&
dataRow.GetData(p.columns.typeColumnName) != alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER] {
@@ -50,23 +51,23 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
data := make(map[datatable.TransactionDataTableColumn]string, len(alipayTransactionSupportedColumns))
if dataTable.HasOriginalColumn(p.columns.timeColumnName) {
if p.hasOriginalColumn(p.columns.timeColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = dataRow.GetData(p.columns.timeColumnName)
}
if dataTable.HasOriginalColumn(p.columns.categoryColumnName) {
if p.hasOriginalColumn(p.columns.categoryColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(p.columns.categoryColumnName)
} else {
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = ""
}
if dataTable.HasOriginalColumn(p.columns.amountColumnName) {
if p.hasOriginalColumn(p.columns.amountColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = dataRow.GetData(p.columns.amountColumnName)
}
if dataTable.HasOriginalColumn(p.columns.descriptionColumnName) && dataRow.GetData(p.columns.descriptionColumnName) != "" {
if p.hasOriginalColumn(p.columns.descriptionColumnName) && dataRow.GetData(p.columns.descriptionColumnName) != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(p.columns.descriptionColumnName)
} else if dataTable.HasOriginalColumn(p.columns.productNameColumnName) && dataRow.GetData(p.columns.productNameColumnName) != "" {
} else if p.hasOriginalColumn(p.columns.productNameColumnName) && dataRow.GetData(p.columns.productNameColumnName) != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(p.columns.productNameColumnName)
} else {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = ""
@@ -74,13 +75,13 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
relatedAccountName := ""
if dataTable.HasOriginalColumn(p.columns.relatedAccountColumnName) {
if p.hasOriginalColumn(p.columns.relatedAccountColumnName) {
relatedAccountName = dataRow.GetData(p.columns.relatedAccountColumnName)
}
statusName := ""
if dataTable.HasOriginalColumn(p.columns.statusColumnName) {
if p.hasOriginalColumn(p.columns.statusColumnName) {
statusName = dataRow.GetData(p.columns.statusColumnName)
}
@@ -92,7 +93,7 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
localeTextItems := locales.GetLocaleTextItems(locale)
if dataTable.HasOriginalColumn(p.columns.typeColumnName) {
if p.hasOriginalColumn(p.columns.typeColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = dataRow.GetData(p.columns.typeColumnName)
if dataRow.GetData(p.columns.typeColumnName) == alipayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] {
@@ -117,11 +118,11 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
targetName := ""
productName := ""
if dataTable.HasOriginalColumn(p.columns.targetNameColumnName) {
if p.hasOriginalColumn(p.columns.targetNameColumnName) {
targetName = dataRow.GetData(p.columns.targetNameColumnName)
}
if dataTable.HasOriginalColumn(p.columns.productNameColumnName) {
if p.hasOriginalColumn(p.columns.productNameColumnName) {
productName = dataRow.GetData(p.columns.productNameColumnName)
}
@@ -170,9 +171,21 @@ func (p *alipayTransactionDataRowParser) Parse(ctx core.Context, user *models.Us
return data, true, nil
}
func (p *alipayTransactionDataRowParser) hasOriginalColumn(columnName string) bool {
_, exists := p.existedOriginalDataColumns[columnName]
return exists
}
// createAlipayTransactionDataRowParser returns alipay transaction data row parser
func createAlipayTransactionDataRowParser(originalColumnNames alipayTransactionColumnNames) datatable.CommonTransactionDataRowParser {
func createAlipayTransactionDataRowParser(originalColumnNames alipayTransactionColumnNames, headerColumnNames []string) datatable.CommonTransactionDataRowParser {
existedOriginalDataColumns := make(map[string]bool, len(headerColumnNames))
for i := 0; i < len(headerColumnNames); i++ {
existedOriginalDataColumns[headerColumnNames[i]] = true
}
return &alipayTransactionDataRowParser{
columns: originalColumnNames,
columns: originalColumnNames,
existedOriginalDataColumns: existedOriginalDataColumns,
}
}
+25 -25
View File
@@ -41,49 +41,49 @@ const (
// beancountData defines the structure of beancount data
type beancountData struct {
accounts map[string]*beancountAccount
transactions []*beancountTransactionEntry
Accounts map[string]*beancountAccount
Transactions []*beancountTransactionEntry
}
// beancountAccount defines the structure of beancount account
type beancountAccount struct {
name string
accountType beancountAccountType
openDate string
closeDate string
Name string
AccountType beancountAccountType
OpenDate string
CloseDate string
}
// beancountTransactionEntry defines the structure of beancount transaction entry
type beancountTransactionEntry struct {
date string
directive beancountDirective
payee string
narration string
postings []*beancountPosting
tags []string
links []string
metadata map[string]string
Date string
Directive beancountDirective
Payee string
Narration string
Postings []*beancountPosting
Tags []string
Links []string
Metadata map[string]string
}
// beancountPosting defines the structure of beancount transaction posting
type beancountPosting struct {
account string
amount string
originalAmount string
commodity string
totalCost string
totalCostCommodity string
price string
priceCommodity string
metadata map[string]string
Account string
Amount string
OriginalAmount string
Commodity string
TotalCost string
TotalCostCommodity string
Price string
PriceCommodity string
Metadata map[string]string
}
func (a *beancountAccount) isOpeningBalanceEquityAccount() bool {
if a.accountType != beancountEquityAccountType {
if a.AccountType != beancountEquityAccountType {
return false
}
nameItems := strings.Split(a.name, string(beancountMetadataKeySuffix))
nameItems := strings.Split(a.Name, string(beancountMetadataKeySuffix))
if len(nameItems) != 2 {
return false
@@ -49,8 +49,8 @@ func (r *beancountDataReader) read(ctx core.Context) (*beancountData, error) {
}
data := &beancountData{
accounts: make(map[string]*beancountAccount),
transactions: make([]*beancountTransactionEntry, 0),
Accounts: make(map[string]*beancountAccount),
Transactions: make([]*beancountTransactionEntry, 0),
}
var err error
@@ -100,7 +100,7 @@ func (r *beancountDataReader) read(ctx core.Context) (*beancountData, error) {
if ('A' <= actualFirstItem[0] && actualFirstItem[0] <= 'Z') || actualFirstItem[0] == '!' { // transaction posting
if currentTransactionEntry != nil && currentTransactionPosting != nil {
currentTransactionEntry.postings = append(currentTransactionEntry.postings, currentTransactionPosting)
currentTransactionEntry.Postings = append(currentTransactionEntry.Postings, currentTransactionPosting)
currentTransactionPosting = nil
}
@@ -120,12 +120,12 @@ func (r *beancountDataReader) read(ctx core.Context) (*beancountData, error) {
metadataValue := metadata[1]
if currentTransactionPosting != nil {
if _, exists := currentTransactionPosting.metadata[metadataKey]; !exists {
currentTransactionPosting.metadata[metadataKey] = metadataValue
if _, exists := currentTransactionPosting.Metadata[metadataKey]; !exists {
currentTransactionPosting.Metadata[metadataKey] = metadataValue
}
} else if currentTransactionEntry != nil {
if _, exists := currentTransactionEntry.metadata[metadataKey]; !exists {
currentTransactionEntry.metadata[metadataKey] = metadataValue
if _, exists := currentTransactionEntry.Metadata[metadataKey]; !exists {
currentTransactionEntry.Metadata[metadataKey] = metadataValue
}
}
} else {
@@ -172,11 +172,11 @@ func (r *beancountDataReader) read(ctx core.Context) (*beancountData, error) {
if currentTransactionEntry != nil {
if currentTransactionPosting != nil {
currentTransactionEntry.postings = append(currentTransactionEntry.postings, currentTransactionPosting)
currentTransactionEntry.Postings = append(currentTransactionEntry.Postings, currentTransactionPosting)
currentTransactionPosting = nil
}
data.transactions = append(data.transactions, currentTransactionEntry)
data.Transactions = append(data.Transactions, currentTransactionEntry)
currentTransactionEntry = nil
}
@@ -186,11 +186,11 @@ func (r *beancountDataReader) read(ctx core.Context) (*beancountData, error) {
func (r *beancountDataReader) updateCurrentState(data *beancountData, currentTransactionEntry *beancountTransactionEntry, currentTransactionPosting *beancountPosting) (*beancountTransactionEntry, *beancountPosting) {
if currentTransactionEntry != nil {
if currentTransactionPosting != nil {
currentTransactionEntry.postings = append(currentTransactionEntry.postings, currentTransactionPosting)
currentTransactionEntry.Postings = append(currentTransactionEntry.Postings, currentTransactionPosting)
currentTransactionPosting = nil
}
data.transactions = append(data.transactions, currentTransactionEntry)
data.Transactions = append(data.Transactions, currentTransactionEntry)
currentTransactionEntry = nil
currentTransactionPosting = nil
}
@@ -277,7 +277,7 @@ func (r *beancountDataReader) readAccountLine(ctx core.Context, lineIndex int, i
var err error
accountName := r.getNotEmptyItemByIndex(items, 2)
account, exists := data.accounts[accountName]
account, exists := data.Accounts[accountName]
if !exists {
account, err = r.createAccount(ctx, data, accountName)
@@ -288,10 +288,10 @@ func (r *beancountDataReader) readAccountLine(ctx core.Context, lineIndex int, i
}
if directive == beancountDirectiveOpen {
account.openDate = date
account.OpenDate = date
return account, nil
} else if directive == beancountDirectiveClose {
account.closeDate = date
account.CloseDate = date
return account, nil
} else {
log.Warnf(ctx, "[beancount_data_reader.parseAccount] cannot parse account line#%d \"%s\", because directive is invalid", lineIndex, strings.Join(items, " "))
@@ -301,8 +301,8 @@ func (r *beancountDataReader) readAccountLine(ctx core.Context, lineIndex int, i
func (r *beancountDataReader) createAccount(ctx core.Context, data *beancountData, accountName string) (*beancountAccount, error) {
account := &beancountAccount{
name: accountName,
accountType: beancountUnknownAccountType,
Name: accountName,
AccountType: beancountUnknownAccountType,
}
accountNameItems := strings.Split(accountName, beancountAccountNameItemsSeparator)
@@ -311,31 +311,31 @@ func (r *beancountDataReader) createAccount(ctx core.Context, data *beancountDat
accountType, exists := r.accountTypeNameMap[accountNameItems[0]]
if exists {
account.accountType = accountType
account.AccountType = accountType
} else {
log.Warnf(ctx, "[beancount_data_reader.createAccount] cannot parse account \"%s\", because account type \"%s\" is invalid", accountName, accountNameItems[0])
return nil, errs.ErrInvalidBeancountFile
}
}
data.accounts[accountName] = account
data.Accounts[accountName] = account
return account, nil
}
func (r *beancountDataReader) readTransactionLine(ctx core.Context, lineIndex int, items []string, date string, directive beancountDirective, tags []string) *beancountTransactionEntry {
transactionEntry := &beancountTransactionEntry{
date: date,
directive: directive,
tags: make([]string, 0),
links: make([]string, 0),
metadata: make(map[string]string),
Date: date,
Directive: directive,
Tags: make([]string, 0),
Links: make([]string, 0),
Metadata: make(map[string]string),
}
transactionEntry.tags = append(transactionEntry.tags, tags...)
transactionEntry.Tags = append(transactionEntry.Tags, tags...)
allTags := make(map[string]bool, len(transactionEntry.tags))
allTags := make(map[string]bool, len(transactionEntry.Tags))
for _, tag := range transactionEntry.tags {
for _, tag := range transactionEntry.Tags {
allTags[tag] = true
}
@@ -363,7 +363,7 @@ func (r *beancountDataReader) readTransactionLine(ctx core.Context, lineIndex in
tagName := item[1:]
if _, exists := allTags[tagName]; !exists {
transactionEntry.tags = append(transactionEntry.tags, tagName)
transactionEntry.Tags = append(transactionEntry.Tags, tagName)
allTags[tagName] = true
}
@@ -371,7 +371,7 @@ func (r *beancountDataReader) readTransactionLine(ctx core.Context, lineIndex in
payeeNarrationLastIndex = i - 1
}
} else if item[0] == beancountLinkPrefix { // [ˆlink]
transactionEntry.links = append(transactionEntry.links, item[1:])
transactionEntry.Links = append(transactionEntry.Links, item[1:])
if i-1 < payeeNarrationLastIndex {
payeeNarrationLastIndex = i - 1
@@ -380,10 +380,10 @@ func (r *beancountDataReader) readTransactionLine(ctx core.Context, lineIndex in
}
if payeeNarrationLastIndex-payeeNarrationFirstIndex >= 1 {
transactionEntry.payee = items[payeeNarrationFirstIndex]
transactionEntry.narration = items[payeeNarrationFirstIndex+1]
transactionEntry.Payee = items[payeeNarrationFirstIndex]
transactionEntry.Narration = items[payeeNarrationFirstIndex+1]
} else if payeeNarrationLastIndex-payeeNarrationFirstIndex >= 0 {
transactionEntry.narration = items[payeeNarrationFirstIndex]
transactionEntry.Narration = items[payeeNarrationFirstIndex]
}
return transactionEntry
@@ -410,36 +410,36 @@ func (r *beancountDataReader) readTransactionPostingLine(ctx core.Context, lineI
}
transactionPositing := &beancountPosting{
account: accountName,
metadata: make(map[string]string),
Account: accountName,
Metadata: make(map[string]string),
}
amountActualLastIndex := -1
transactionPositing.originalAmount, amountActualLastIndex = r.getOriginalAmountAndLastIndexFromIndex(items, accountNameActualIndex+1)
transactionPositing.OriginalAmount, amountActualLastIndex = r.getOriginalAmountAndLastIndexFromIndex(items, accountNameActualIndex+1)
if transactionPositing.originalAmount == "" || amountActualLastIndex < 0 {
if transactionPositing.OriginalAmount == "" || amountActualLastIndex < 0 {
log.Warnf(ctx, "[beancount_data_reader.readTransactionPostingLine] cannot parse transaction posting line#%d \"%s\", because missing amount", lineIndex, strings.Join(items, " "))
return nil, errs.ErrAmountInvalid
}
finalAmount, err := evaluateBeancountAmountExpression(ctx, transactionPositing.originalAmount)
finalAmount, err := evaluateBeancountAmountExpression(ctx, transactionPositing.OriginalAmount)
if err != nil {
log.Warnf(ctx, "[beancount_data_reader.readTransactionPostingLine] cannot evaluate amount expression in line#%d \"%s\", because %s", lineIndex, strings.Join(items, " "), err.Error())
return nil, errs.ErrAmountInvalid
} else {
transactionPositing.amount = finalAmount
transactionPositing.Amount = finalAmount
}
commodityActualIndex := -1
transactionPositing.commodity, commodityActualIndex = r.getNotEmptyItemAndIndexFromIndex(items, amountActualLastIndex+1)
transactionPositing.Commodity, commodityActualIndex = r.getNotEmptyItemAndIndexFromIndex(items, amountActualLastIndex+1)
if transactionPositing.commodity == "" || commodityActualIndex < 0 {
if transactionPositing.Commodity == "" || commodityActualIndex < 0 {
log.Warnf(ctx, "[beancount_data_reader.readTransactionPostingLine] cannot parse transaction posting line#%d \"%s\", because missing commodity", lineIndex, strings.Join(items, " "))
return nil, errs.ErrInvalidBeancountFile
}
if strings.ToUpper(transactionPositing.commodity) != transactionPositing.commodity { // The syntax for a currency is a word all in capital letters
if strings.ToUpper(transactionPositing.Commodity) != transactionPositing.Commodity { // The syntax for a currency is a word all in capital letters
log.Warnf(ctx, "[beancount_data_reader.readTransactionPostingLine] cannot parse transaction posting line#%d \"%s\", because commodity name is not capital letters", lineIndex, strings.Join(items, " "))
return nil, errs.ErrInvalidBeancountFile
}
@@ -461,13 +461,13 @@ func (r *beancountDataReader) readTransactionPostingLine(ctx core.Context, lineI
totalCost, totalCostActualIndex := r.getNotEmptyItemAndIndexFromIndex(items, i+1)
if totalCostActualIndex > 0 {
transactionPositing.totalCost = totalCost
transactionPositing.TotalCost = totalCost
i = totalCostActualIndex
totalCostCommodity, totalCostCommodityActualIndex := r.getNotEmptyItemAndIndexFromIndex(items, totalCostActualIndex+1)
if totalCostCommodityActualIndex > 0 {
transactionPositing.totalCostCommodity = totalCostCommodity
transactionPositing.TotalCostCommodity = totalCostCommodity
i = totalCostCommodityActualIndex
}
}
@@ -475,13 +475,13 @@ func (r *beancountDataReader) readTransactionPostingLine(ctx core.Context, lineI
price, priceActualIndex := r.getNotEmptyItemAndIndexFromIndex(items, i+1)
if priceActualIndex > 0 {
transactionPositing.price = price
transactionPositing.Price = price
i = priceActualIndex
priceCommodity, priceCommodityActualIndex := r.getNotEmptyItemAndIndexFromIndex(items, priceActualIndex+1)
if priceCommodityActualIndex > 0 {
transactionPositing.priceCommodity = priceCommodity
transactionPositing.PriceCommodity = priceCommodity
i = priceCommodityActualIndex
}
}
@@ -489,11 +489,11 @@ func (r *beancountDataReader) readTransactionPostingLine(ctx core.Context, lineI
}
}
if transactionPositing.account != "" {
_, exists := data.accounts[transactionPositing.account]
if transactionPositing.Account != "" {
_, exists := data.Accounts[transactionPositing.Account]
if !exists {
_, err := r.createAccount(ctx, data, transactionPositing.account)
_, err := r.createAccount(ctx, data, transactionPositing.Account)
if err != nil {
return nil, err
@@ -41,56 +41,56 @@ func TestBeancountDataReaderRead(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 5, len(actualData.accounts))
assert.Equal(t, "AssetsAccount:TestAccount", actualData.accounts["AssetsAccount:TestAccount"].name)
assert.Equal(t, beancountAssetsAccountType, actualData.accounts["AssetsAccount:TestAccount"].accountType)
assert.Equal(t, "2024-01-01", actualData.accounts["AssetsAccount:TestAccount"].openDate)
assert.Equal(t, "2024-01-07", actualData.accounts["AssetsAccount:TestAccount"].closeDate)
assert.Equal(t, 5, len(actualData.Accounts))
assert.Equal(t, "AssetsAccount:TestAccount", actualData.Accounts["AssetsAccount:TestAccount"].Name)
assert.Equal(t, beancountAssetsAccountType, actualData.Accounts["AssetsAccount:TestAccount"].AccountType)
assert.Equal(t, "2024-01-01", actualData.Accounts["AssetsAccount:TestAccount"].OpenDate)
assert.Equal(t, "2024-01-07", actualData.Accounts["AssetsAccount:TestAccount"].CloseDate)
assert.Equal(t, "LiabilitiesAccount:TestAccount2", actualData.accounts["LiabilitiesAccount:TestAccount2"].name)
assert.Equal(t, beancountLiabilitiesAccountType, actualData.accounts["LiabilitiesAccount:TestAccount2"].accountType)
assert.Equal(t, "2024-01-02", actualData.accounts["LiabilitiesAccount:TestAccount2"].openDate)
assert.Equal(t, "LiabilitiesAccount:TestAccount2", actualData.Accounts["LiabilitiesAccount:TestAccount2"].Name)
assert.Equal(t, beancountLiabilitiesAccountType, actualData.Accounts["LiabilitiesAccount:TestAccount2"].AccountType)
assert.Equal(t, "2024-01-02", actualData.Accounts["LiabilitiesAccount:TestAccount2"].OpenDate)
assert.Equal(t, 2, len(actualData.transactions))
assert.Equal(t, 2, len(actualData.Transactions))
assert.Equal(t, "2024-01-05", actualData.transactions[0].date)
assert.Equal(t, "Payee Name", actualData.transactions[0].payee)
assert.Equal(t, "Foo Bar", actualData.transactions[0].narration)
assert.Equal(t, 2, len(actualData.transactions[0].postings))
assert.Equal(t, "IncomeAccount:TestCategory", actualData.transactions[0].postings[0].account)
assert.Equal(t, "-123.45", actualData.transactions[0].postings[0].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[0].commodity)
assert.Equal(t, "AssetsAccount:TestAccount", actualData.transactions[0].postings[1].account)
assert.Equal(t, "123.45", actualData.transactions[0].postings[1].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[1].commodity)
assert.Equal(t, "2024-01-05", actualData.Transactions[0].Date)
assert.Equal(t, "Payee Name", actualData.Transactions[0].Payee)
assert.Equal(t, "Foo Bar", actualData.Transactions[0].Narration)
assert.Equal(t, 2, len(actualData.Transactions[0].Postings))
assert.Equal(t, "IncomeAccount:TestCategory", actualData.Transactions[0].Postings[0].Account)
assert.Equal(t, "-123.45", actualData.Transactions[0].Postings[0].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[0].Commodity)
assert.Equal(t, "AssetsAccount:TestAccount", actualData.Transactions[0].Postings[1].Account)
assert.Equal(t, "123.45", actualData.Transactions[0].Postings[1].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[1].Commodity)
assert.Equal(t, 4, len(actualData.transactions[0].tags))
assert.Equal(t, actualData.transactions[0].tags[0], "tag1")
assert.Equal(t, actualData.transactions[0].tags[1], "tag2")
assert.Equal(t, actualData.transactions[0].tags[2], "tag3")
assert.Equal(t, actualData.transactions[0].tags[3], "tag4")
assert.Equal(t, 4, len(actualData.Transactions[0].Tags))
assert.Equal(t, actualData.Transactions[0].Tags[0], "tag1")
assert.Equal(t, actualData.Transactions[0].Tags[1], "tag2")
assert.Equal(t, actualData.Transactions[0].Tags[2], "tag3")
assert.Equal(t, actualData.Transactions[0].Tags[3], "tag4")
assert.Equal(t, 1, len(actualData.transactions[0].links))
assert.Equal(t, actualData.transactions[0].links[0], "test-link")
assert.Equal(t, 1, len(actualData.Transactions[0].Links))
assert.Equal(t, actualData.Transactions[0].Links[0], "test-link")
assert.Equal(t, "2024-01-06", actualData.transactions[1].date)
assert.Equal(t, "", actualData.transactions[1].payee)
assert.Equal(t, "test\n#test2", actualData.transactions[1].narration)
assert.Equal(t, 2, len(actualData.transactions[1].postings))
assert.Equal(t, "LiabilitiesAccount:TestAccount2", actualData.transactions[1].postings[0].account)
assert.Equal(t, "-0.12", actualData.transactions[1].postings[0].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[0].commodity)
assert.Equal(t, "ExpensesAccount:TestCategory2", actualData.transactions[1].postings[1].account)
assert.Equal(t, "0.12", actualData.transactions[1].postings[1].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[1].commodity)
assert.Equal(t, "2024-01-06", actualData.Transactions[1].Date)
assert.Equal(t, "", actualData.Transactions[1].Payee)
assert.Equal(t, "test\n#test2", actualData.Transactions[1].Narration)
assert.Equal(t, 2, len(actualData.Transactions[1].Postings))
assert.Equal(t, "LiabilitiesAccount:TestAccount2", actualData.Transactions[1].Postings[0].Account)
assert.Equal(t, "-0.12", actualData.Transactions[1].Postings[0].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[0].Commodity)
assert.Equal(t, "ExpensesAccount:TestCategory2", actualData.Transactions[1].Postings[1].Account)
assert.Equal(t, "0.12", actualData.Transactions[1].Postings[1].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[1].Commodity)
assert.Equal(t, 3, len(actualData.transactions[1].tags))
assert.Equal(t, actualData.transactions[1].tags[0], "tag2")
assert.Equal(t, actualData.transactions[1].tags[1], "tag5")
assert.Equal(t, actualData.transactions[1].tags[2], "tag6")
assert.Equal(t, 3, len(actualData.Transactions[1].Tags))
assert.Equal(t, actualData.Transactions[1].Tags[0], "tag2")
assert.Equal(t, actualData.Transactions[1].Tags[1], "tag5")
assert.Equal(t, actualData.Transactions[1].Tags[2], "tag6")
assert.Equal(t, 1, len(actualData.transactions[1].links))
assert.Equal(t, actualData.transactions[1].links[0], "test-link2")
assert.Equal(t, 1, len(actualData.Transactions[1].Links))
assert.Equal(t, actualData.Transactions[1].Links[0], "test-link2")
}
func TestBeancountDataReaderRead_EmptyContent(t *testing.T) {
@@ -147,17 +147,17 @@ func TestBeancountDataReaderReadAndSetOption_AccountTypeName(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 3, len(actualData.accounts))
assert.Equal(t, 3, len(actualData.Accounts))
assert.Equal(t, "A:TestAccount", actualData.accounts["A:TestAccount"].name)
assert.Equal(t, beancountAssetsAccountType, actualData.accounts["A:TestAccount"].accountType)
assert.Equal(t, "A:TestAccount", actualData.Accounts["A:TestAccount"].Name)
assert.Equal(t, beancountAssetsAccountType, actualData.Accounts["A:TestAccount"].AccountType)
assert.Equal(t, "L:TestAccount2", actualData.accounts["L:TestAccount2"].name)
assert.Equal(t, beancountLiabilitiesAccountType, actualData.accounts["L:TestAccount2"].accountType)
assert.Equal(t, "L:TestAccount2", actualData.Accounts["L:TestAccount2"].Name)
assert.Equal(t, beancountLiabilitiesAccountType, actualData.Accounts["L:TestAccount2"].AccountType)
assert.Equal(t, "E:Opening-Balances", actualData.accounts["E:Opening-Balances"].name)
assert.Equal(t, beancountEquityAccountType, actualData.accounts["E:Opening-Balances"].accountType)
assert.True(t, actualData.accounts["E:Opening-Balances"].isOpeningBalanceEquityAccount())
assert.Equal(t, "E:Opening-Balances", actualData.Accounts["E:Opening-Balances"].Name)
assert.Equal(t, beancountEquityAccountType, actualData.Accounts["E:Opening-Balances"].AccountType)
assert.True(t, actualData.Accounts["E:Opening-Balances"].isOpeningBalanceEquityAccount())
}
func TestBeancountDataReaderReadAndSetOption_InvalidLineOrUnsupportedOption(t *testing.T) {
@@ -203,31 +203,31 @@ func TestBeancountDataReaderReadAndSetTags(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 5, len(actualData.transactions))
assert.Equal(t, 5, len(actualData.Transactions))
assert.Equal(t, 4, len(actualData.transactions[0].tags))
assert.Equal(t, actualData.transactions[0].tags[0], "tag1")
assert.Equal(t, actualData.transactions[0].tags[1], "tag2")
assert.Equal(t, actualData.transactions[0].tags[2], "tag3")
assert.Equal(t, actualData.transactions[0].tags[3], "tag4")
assert.Equal(t, 4, len(actualData.Transactions[0].Tags))
assert.Equal(t, actualData.Transactions[0].Tags[0], "tag1")
assert.Equal(t, actualData.Transactions[0].Tags[1], "tag2")
assert.Equal(t, actualData.Transactions[0].Tags[2], "tag3")
assert.Equal(t, actualData.Transactions[0].Tags[3], "tag4")
assert.Equal(t, 2, len(actualData.transactions[1].tags))
assert.Equal(t, actualData.transactions[1].tags[0], "tag5")
assert.Equal(t, actualData.transactions[1].tags[1], "tag6")
assert.Equal(t, 2, len(actualData.Transactions[1].Tags))
assert.Equal(t, actualData.Transactions[1].Tags[0], "tag5")
assert.Equal(t, actualData.Transactions[1].Tags[1], "tag6")
assert.Equal(t, 2, len(actualData.transactions[2].tags))
assert.Equal(t, actualData.transactions[2].tags[0], "tag5")
assert.Equal(t, actualData.transactions[2].tags[1], "tag6")
assert.Equal(t, 2, len(actualData.Transactions[2].Tags))
assert.Equal(t, actualData.Transactions[2].Tags[0], "tag5")
assert.Equal(t, actualData.Transactions[2].Tags[1], "tag6")
assert.Equal(t, 3, len(actualData.transactions[3].tags))
assert.Equal(t, actualData.transactions[3].tags[0], "tag3")
assert.Equal(t, actualData.transactions[3].tags[1], "tag6")
assert.Equal(t, actualData.transactions[3].tags[2], "tag5")
assert.Equal(t, 3, len(actualData.Transactions[3].Tags))
assert.Equal(t, actualData.Transactions[3].Tags[0], "tag3")
assert.Equal(t, actualData.Transactions[3].Tags[1], "tag6")
assert.Equal(t, actualData.Transactions[3].Tags[2], "tag5")
assert.Equal(t, 3, len(actualData.transactions[4].tags))
assert.Equal(t, actualData.transactions[4].tags[0], "tag3")
assert.Equal(t, actualData.transactions[4].tags[1], "tag6")
assert.Equal(t, actualData.transactions[4].tags[2], "tag5")
assert.Equal(t, 3, len(actualData.Transactions[4].Tags))
assert.Equal(t, actualData.Transactions[4].Tags[0], "tag3")
assert.Equal(t, actualData.Transactions[4].Tags[1], "tag6")
assert.Equal(t, actualData.Transactions[4].Tags[2], "tag5")
}
func TestBeancountDataReaderReadAccountLine_InvalidLine(t *testing.T) {
@@ -238,7 +238,7 @@ func TestBeancountDataReaderReadAccountLine_InvalidLine(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 0, len(actualData.accounts))
assert.Equal(t, 0, len(actualData.Accounts))
}
func TestBeancountDataReaderReadAccountLine_InvalidAccountType(t *testing.T) {
@@ -274,44 +274,44 @@ func TestBeancountDataReaderReadTransactionLine(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 6, len(actualData.transactions))
assert.Equal(t, 6, len(actualData.Transactions))
assert.Equal(t, "2024-01-01", actualData.transactions[0].date)
assert.Equal(t, beancountDirectiveCompletedTransaction, actualData.transactions[0].directive)
assert.Equal(t, "", actualData.transactions[0].payee)
assert.Equal(t, "", actualData.transactions[0].narration)
assert.Equal(t, "2024-01-01", actualData.Transactions[0].Date)
assert.Equal(t, beancountDirectiveCompletedTransaction, actualData.Transactions[0].Directive)
assert.Equal(t, "", actualData.Transactions[0].Payee)
assert.Equal(t, "", actualData.Transactions[0].Narration)
assert.Equal(t, "2024-01-02", actualData.transactions[1].date)
assert.Equal(t, beancountDirectiveCompletedTransaction, actualData.transactions[1].directive)
assert.Equal(t, "", actualData.transactions[1].payee)
assert.Equal(t, "test\ttest2\ntest3", actualData.transactions[1].narration)
assert.Equal(t, "2024-01-02", actualData.Transactions[1].Date)
assert.Equal(t, beancountDirectiveCompletedTransaction, actualData.Transactions[1].Directive)
assert.Equal(t, "", actualData.Transactions[1].Payee)
assert.Equal(t, "test\ttest2\ntest3", actualData.Transactions[1].Narration)
assert.Equal(t, "2024-01-03", actualData.transactions[2].date)
assert.Equal(t, beancountDirectiveInCompleteTransaction, actualData.transactions[2].directive)
assert.Equal(t, "test", actualData.transactions[2].payee)
assert.Equal(t, "test2", actualData.transactions[2].narration)
assert.Equal(t, "2024-01-03", actualData.Transactions[2].Date)
assert.Equal(t, beancountDirectiveInCompleteTransaction, actualData.Transactions[2].Directive)
assert.Equal(t, "test", actualData.Transactions[2].Payee)
assert.Equal(t, "test2", actualData.Transactions[2].Narration)
assert.Equal(t, "2024-01-04", actualData.transactions[3].date)
assert.Equal(t, beancountDirectivePaddingTransaction, actualData.transactions[3].directive)
assert.Equal(t, "", actualData.transactions[3].payee)
assert.Equal(t, "test", actualData.transactions[3].narration)
assert.Equal(t, "2024-01-04", actualData.Transactions[3].Date)
assert.Equal(t, beancountDirectivePaddingTransaction, actualData.Transactions[3].Directive)
assert.Equal(t, "", actualData.Transactions[3].Payee)
assert.Equal(t, "test", actualData.Transactions[3].Narration)
assert.Equal(t, 2, len(actualData.transactions[3].tags))
assert.Equal(t, actualData.transactions[3].tags[0], "tag")
assert.Equal(t, actualData.transactions[3].tags[1], "tag2")
assert.Equal(t, 2, len(actualData.Transactions[3].Tags))
assert.Equal(t, actualData.Transactions[3].Tags[0], "tag")
assert.Equal(t, actualData.Transactions[3].Tags[1], "tag2")
assert.Equal(t, "2024-01-05", actualData.transactions[4].date)
assert.Equal(t, beancountDirectiveTransaction, actualData.transactions[4].directive)
assert.Equal(t, "", actualData.transactions[4].payee)
assert.Equal(t, "test", actualData.transactions[4].narration)
assert.Equal(t, "2024-01-05", actualData.Transactions[4].Date)
assert.Equal(t, beancountDirectiveTransaction, actualData.Transactions[4].Directive)
assert.Equal(t, "", actualData.Transactions[4].Payee)
assert.Equal(t, "test", actualData.Transactions[4].Narration)
assert.Equal(t, 1, len(actualData.transactions[4].links))
assert.Equal(t, actualData.transactions[4].links[0], "scheme://path/to/test/link")
assert.Equal(t, 1, len(actualData.Transactions[4].Links))
assert.Equal(t, actualData.Transactions[4].Links[0], "scheme://path/to/test/link")
assert.Equal(t, "2024-01-06", actualData.transactions[5].date)
assert.Equal(t, beancountDirectiveTransaction, actualData.transactions[5].directive)
assert.Equal(t, "", actualData.transactions[5].payee)
assert.Equal(t, "", actualData.transactions[5].narration)
assert.Equal(t, "2024-01-06", actualData.Transactions[5].Date)
assert.Equal(t, beancountDirectiveTransaction, actualData.Transactions[5].Directive)
assert.Equal(t, "", actualData.Transactions[5].Payee)
assert.Equal(t, "", actualData.Transactions[5].Narration)
}
func TestBeancountDataReaderReadTransactionPostingLine(t *testing.T) {
@@ -331,39 +331,39 @@ func TestBeancountDataReaderReadTransactionPostingLine(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 2, len(actualData.transactions))
assert.Equal(t, 2, len(actualData.Transactions))
assert.Equal(t, "2024-01-01", actualData.transactions[0].date)
assert.Equal(t, 2, len(actualData.transactions[0].postings))
assert.Equal(t, "Income:TestCategory", actualData.transactions[0].postings[0].account)
assert.Equal(t, "-123.45", actualData.transactions[0].postings[0].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[0].commodity)
assert.Equal(t, "2024-01-01", actualData.Transactions[0].Date)
assert.Equal(t, 2, len(actualData.Transactions[0].Postings))
assert.Equal(t, "Income:TestCategory", actualData.Transactions[0].Postings[0].Account)
assert.Equal(t, "-123.45", actualData.Transactions[0].Postings[0].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[0].Commodity)
assert.Equal(t, "Assets:TestAccount", actualData.transactions[0].postings[1].account)
assert.Equal(t, "123.45", actualData.transactions[0].postings[1].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[1].commodity)
assert.Equal(t, "Assets:TestAccount", actualData.Transactions[0].Postings[1].Account)
assert.Equal(t, "123.45", actualData.Transactions[0].Postings[1].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[1].Commodity)
assert.Equal(t, "2024-01-02", actualData.transactions[1].date)
assert.Equal(t, 4, len(actualData.transactions[1].postings))
assert.Equal(t, "2024-01-02", actualData.Transactions[1].Date)
assert.Equal(t, 4, len(actualData.Transactions[1].Postings))
assert.Equal(t, "Liabilities:TestAccount2", actualData.transactions[1].postings[0].account)
assert.Equal(t, "-0.23", actualData.transactions[1].postings[0].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[0].commodity)
assert.Equal(t, "Expenses:TestCategory2", actualData.transactions[1].postings[1].account)
assert.Equal(t, "Liabilities:TestAccount2", actualData.Transactions[1].Postings[0].Account)
assert.Equal(t, "-0.23", actualData.Transactions[1].Postings[0].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[0].Commodity)
assert.Equal(t, "Expenses:TestCategory2", actualData.Transactions[1].Postings[1].Account)
assert.Equal(t, "0.12", actualData.transactions[1].postings[1].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[1].commodity)
assert.Equal(t, "0.84", actualData.transactions[1].postings[1].totalCost)
assert.Equal(t, "CNY", actualData.transactions[1].postings[1].totalCostCommodity)
assert.Equal(t, "Expenses:TestCategory3", actualData.transactions[1].postings[2].account)
assert.Equal(t, "0.12", actualData.Transactions[1].Postings[1].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[1].Commodity)
assert.Equal(t, "0.84", actualData.Transactions[1].Postings[1].TotalCost)
assert.Equal(t, "CNY", actualData.Transactions[1].Postings[1].TotalCostCommodity)
assert.Equal(t, "Expenses:TestCategory3", actualData.Transactions[1].Postings[2].Account)
assert.Equal(t, "0.11", actualData.transactions[1].postings[2].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[2].commodity)
assert.Equal(t, "7.12", actualData.transactions[1].postings[2].price)
assert.Equal(t, "CNY", actualData.transactions[1].postings[2].priceCommodity)
assert.Equal(t, "0.11", actualData.Transactions[1].Postings[2].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[2].Commodity)
assert.Equal(t, "7.12", actualData.Transactions[1].Postings[2].Price)
assert.Equal(t, "CNY", actualData.Transactions[1].Postings[2].PriceCommodity)
assert.Equal(t, "0.00", actualData.transactions[1].postings[3].amount)
assert.Equal(t, "USD", actualData.transactions[1].postings[3].commodity)
assert.Equal(t, "0.00", actualData.Transactions[1].Postings[3].Amount)
assert.Equal(t, "USD", actualData.Transactions[1].Postings[3].Commodity)
}
func TestBeancountDataReaderReadTransactionPostingLine_AmountExpression(t *testing.T) {
@@ -377,19 +377,19 @@ func TestBeancountDataReaderReadTransactionPostingLine_AmountExpression(t *testi
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.transactions))
assert.Equal(t, 1, len(actualData.Transactions))
assert.Equal(t, "2024-01-01", actualData.transactions[0].date)
assert.Equal(t, 2, len(actualData.transactions[0].postings))
assert.Equal(t, "Income:TestCategory", actualData.transactions[0].postings[0].account)
assert.Equal(t, "(1.2-3.4) * 5.6 / 7.8", actualData.transactions[0].postings[0].originalAmount)
assert.Equal(t, "-1.58", actualData.transactions[0].postings[0].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[0].commodity)
assert.Equal(t, "2024-01-01", actualData.Transactions[0].Date)
assert.Equal(t, 2, len(actualData.Transactions[0].Postings))
assert.Equal(t, "Income:TestCategory", actualData.Transactions[0].Postings[0].Account)
assert.Equal(t, "(1.2-3.4) * 5.6 / 7.8", actualData.Transactions[0].Postings[0].OriginalAmount)
assert.Equal(t, "-1.58", actualData.Transactions[0].Postings[0].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[0].Commodity)
assert.Equal(t, "Assets:TestAccount", actualData.transactions[0].postings[1].account)
assert.Equal(t, "1.2 * 3.4/-5.6 - 7.8", actualData.transactions[0].postings[1].originalAmount)
assert.Equal(t, "-8.53", actualData.transactions[0].postings[1].amount)
assert.Equal(t, "CNY", actualData.transactions[0].postings[1].commodity)
assert.Equal(t, "Assets:TestAccount", actualData.Transactions[0].Postings[1].Account)
assert.Equal(t, "1.2 * 3.4/-5.6 - 7.8", actualData.Transactions[0].Postings[1].OriginalAmount)
assert.Equal(t, "-8.53", actualData.Transactions[0].Postings[1].Amount)
assert.Equal(t, "CNY", actualData.Transactions[0].Postings[1].Commodity)
}
func TestBeancountDataReaderReadTransactionPostingLine_InvalidAmountExpression(t *testing.T) {
@@ -444,8 +444,8 @@ func TestBeancountDataReaderReadTransactionPostingLine_MissingAmount(t *testing.
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.transactions))
assert.Equal(t, 0, len(actualData.transactions[0].postings))
assert.Equal(t, 1, len(actualData.Transactions))
assert.Equal(t, 0, len(actualData.Transactions[0].Postings))
reader, err = createNewBeancountDataReader(context, []byte(""+
"2024-01-01 *\n"+
@@ -454,8 +454,8 @@ func TestBeancountDataReaderReadTransactionPostingLine_MissingAmount(t *testing.
actualData, err = reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.transactions))
assert.Equal(t, 0, len(actualData.transactions[0].postings))
assert.Equal(t, 1, len(actualData.Transactions))
assert.Equal(t, 0, len(actualData.Transactions[0].Postings))
}
func TestBeancountDataReaderReadTransactionPostingLine_MissingCommodity(t *testing.T) {
@@ -503,18 +503,18 @@ func TestBeancountDataReaderReadTransactionMetadataLine(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 2, len(actualData.transactions))
assert.Equal(t, 2, len(actualData.Transactions))
assert.Equal(t, "2024-01-01", actualData.transactions[0].date)
assert.Equal(t, 2, len(actualData.transactions[0].postings))
assert.Equal(t, 2, len(actualData.transactions[0].metadata))
assert.Equal(t, "value", actualData.transactions[0].metadata["key"])
assert.Equal(t, "value 2", actualData.transactions[0].metadata["key2"])
assert.Equal(t, "2024-01-01", actualData.Transactions[0].Date)
assert.Equal(t, 2, len(actualData.Transactions[0].Postings))
assert.Equal(t, 2, len(actualData.Transactions[0].Metadata))
assert.Equal(t, "value", actualData.Transactions[0].Metadata["key"])
assert.Equal(t, "value 2", actualData.Transactions[0].Metadata["key2"])
assert.Equal(t, "2024-01-02", actualData.transactions[1].date)
assert.Equal(t, 2, len(actualData.transactions[1].postings))
assert.Equal(t, 2, len(actualData.transactions[1].postings[0].metadata))
assert.Equal(t, "value6", actualData.transactions[1].postings[0].metadata["key6"])
assert.Equal(t, "value 7", actualData.transactions[1].postings[0].metadata["key7"])
assert.Equal(t, 0, len(actualData.transactions[1].postings[1].metadata))
assert.Equal(t, "2024-01-02", actualData.Transactions[1].Date)
assert.Equal(t, 2, len(actualData.Transactions[1].Postings))
assert.Equal(t, 2, len(actualData.Transactions[1].Postings[0].Metadata))
assert.Equal(t, "value6", actualData.Transactions[1].Postings[0].Metadata["key6"])
assert.Equal(t, "value 7", actualData.Transactions[1].Postings[0].Metadata["key7"])
assert.Equal(t, 0, len(actualData.Transactions[1].Postings[1].Metadata))
}
+10 -10
View File
@@ -8,34 +8,34 @@ import (
func TestBeancountAccount_IsOpeningBalanceEquityAccount_True(t *testing.T) {
account := beancountAccount{
accountType: beancountEquityAccountType,
name: "Equity:Opening-Balances",
AccountType: beancountEquityAccountType,
Name: "Equity:Opening-Balances",
}
assert.True(t, account.isOpeningBalanceEquityAccount())
account = beancountAccount{
accountType: beancountEquityAccountType,
name: "E:Opening-Balances",
AccountType: beancountEquityAccountType,
Name: "E:Opening-Balances",
}
assert.True(t, account.isOpeningBalanceEquityAccount())
}
func TestBeancountAccount_IsOpeningBalanceEquityAccount_False(t *testing.T) {
account := beancountAccount{
accountType: beancountAssetsAccountType,
name: "Equity:Opening-Balances",
AccountType: beancountAssetsAccountType,
Name: "Equity:Opening-Balances",
}
assert.False(t, account.isOpeningBalanceEquityAccount())
account = beancountAccount{
accountType: beancountEquityAccountType,
name: "Opening-Balances",
AccountType: beancountEquityAccountType,
Name: "Opening-Balances",
}
assert.False(t, account.isOpeningBalanceEquityAccount())
account = beancountAccount{
accountType: beancountEquityAccountType,
name: "Equity:Other",
AccountType: beancountEquityAccountType,
Name: "Equity:Other",
}
assert.False(t, account.isOpeningBalanceEquityAccount())
}
@@ -43,7 +43,7 @@ func (c *beancountTransactionDataImporter) ParseImportedData(ctx core.Context, u
return nil, nil, nil, nil, nil, nil, err
}
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(beancountTransactionTypeNameMapping, "", BEANCOUNT_TRANSACTION_TAG_SEPARATOR)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(beancountTransactionTypeNameMapping, "", "", BEANCOUNT_TRANSACTION_TAG_SEPARATOR)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -85,7 +85,7 @@ func (t *beancountTransactionDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allData)
}
// Next returns the next imported data row
// Next returns the next transaction data row
func (t *beancountTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
if t.currentIndex+1 >= len(t.dataTable.allData) {
return nil, nil
@@ -110,49 +110,49 @@ func (t *beancountTransactionDataRowIterator) Next(ctx core.Context, user *model
func (t *beancountTransactionDataRowIterator) parseTransaction(ctx core.Context, user *models.User, beancountEntry *beancountTransactionEntry) (map[datatable.TransactionDataTableColumn]string, error) {
data := make(map[datatable.TransactionDataTableColumn]string, len(beancountTransactionSupportedColumns))
if beancountEntry.date == "" {
if beancountEntry.Date == "" {
return nil, errs.ErrMissingTransactionTime
}
// Beancount supports the international ISO 8601 standard format for dates, with dashes or the same ordering with slashes
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = strings.ReplaceAll(beancountEntry.date, "/", "-") + " 00:00:00"
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = strings.ReplaceAll(beancountEntry.Date, "/", "-") + " 00:00:00"
if len(beancountEntry.postings) == 2 {
splitData1 := beancountEntry.postings[0]
splitData2 := beancountEntry.postings[1]
if len(beancountEntry.Postings) == 2 {
splitData1 := beancountEntry.Postings[0]
splitData2 := beancountEntry.Postings[1]
account1 := t.dataTable.accountMap[splitData1.account]
account2 := t.dataTable.accountMap[splitData2.account]
account1 := t.dataTable.accountMap[splitData1.Account]
account2 := t.dataTable.accountMap[splitData2.Account]
if account1 == nil || account2 == nil {
return nil, errs.ErrMissingAccountData
}
amount1, err := utils.ParseAmount(splitData1.amount)
amount1, err := utils.ParseAmount(splitData1.Amount)
if err != nil {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse amount \"%s\", because %s", splitData1.amount, err.Error())
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse amount \"%s\", because %s", splitData1.Amount, err.Error())
return nil, errs.ErrAmountInvalid
}
amount2, err := utils.ParseAmount(splitData2.amount)
amount2, err := utils.ParseAmount(splitData2.Amount)
if err != nil {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse amount \"%s\", because %s", splitData2.amount, err.Error())
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse amount \"%s\", because %s", splitData2.Amount, err.Error())
return nil, errs.ErrAmountInvalid
}
if ((account1.accountType == beancountEquityAccountType || account1.accountType == beancountIncomeAccountType) && (account2.accountType == beancountAssetsAccountType || account2.accountType == beancountLiabilitiesAccountType)) ||
((account2.accountType == beancountEquityAccountType || account2.accountType == beancountIncomeAccountType) && (account1.accountType == beancountAssetsAccountType || account1.accountType == beancountLiabilitiesAccountType)) { // income
if ((account1.AccountType == beancountEquityAccountType || account1.AccountType == beancountIncomeAccountType) && (account2.AccountType == beancountAssetsAccountType || account2.AccountType == beancountLiabilitiesAccountType)) ||
((account2.AccountType == beancountEquityAccountType || account2.AccountType == beancountIncomeAccountType) && (account1.AccountType == beancountAssetsAccountType || account1.AccountType == beancountLiabilitiesAccountType)) { // income
fromAccount := account1
toAccount := account2
toCurrency := splitData2.commodity
toCurrency := splitData2.Commodity
toAmount := amount2
if (account2.accountType == beancountEquityAccountType || account2.accountType == beancountIncomeAccountType) && (account1.accountType == beancountAssetsAccountType || account1.accountType == beancountLiabilitiesAccountType) {
if (account2.AccountType == beancountEquityAccountType || account2.AccountType == beancountIncomeAccountType) && (account1.AccountType == beancountAssetsAccountType || account1.AccountType == beancountLiabilitiesAccountType) {
fromAccount = account2
toAccount = account1
toCurrency = splitData1.commodity
toCurrency = splitData1.Commodity
toAmount = amount1
}
@@ -162,48 +162,48 @@ func (t *beancountTransactionDataRowIterator) parseTransaction(ctx core.Context,
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_INCOME))
}
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = fromAccount.name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = toAccount.name
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = fromAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = toAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = toCurrency
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(toAmount)
} else if account1.accountType == beancountExpensesAccountType && (account2.accountType == beancountAssetsAccountType || account2.accountType == beancountLiabilitiesAccountType) ||
(account2.accountType == beancountExpensesAccountType && (account1.accountType == beancountAssetsAccountType || account1.accountType == beancountLiabilitiesAccountType)) { // expense
} else if account1.AccountType == beancountExpensesAccountType && (account2.AccountType == beancountAssetsAccountType || account2.AccountType == beancountLiabilitiesAccountType) ||
(account2.AccountType == beancountExpensesAccountType && (account1.AccountType == beancountAssetsAccountType || account1.AccountType == beancountLiabilitiesAccountType)) { // expense
fromAccount := account1
fromCurrency := splitData1.commodity
fromCurrency := splitData1.Commodity
fromAmount := amount1
toAccount := account2
if account1.accountType == beancountExpensesAccountType && (account2.accountType == beancountAssetsAccountType || account2.accountType == beancountLiabilitiesAccountType) {
if account1.AccountType == beancountExpensesAccountType && (account2.AccountType == beancountAssetsAccountType || account2.AccountType == beancountLiabilitiesAccountType) {
fromAccount = account2
fromCurrency = splitData2.commodity
fromCurrency = splitData2.Commodity
fromAmount = amount2
toAccount = account1
}
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE))
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = toAccount.name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = fromAccount.name
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = toAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = fromAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = fromCurrency
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-fromAmount)
} else if (account1.accountType == beancountAssetsAccountType || account1.accountType == beancountLiabilitiesAccountType) &&
(account2.accountType == beancountAssetsAccountType || account2.accountType == beancountLiabilitiesAccountType) {
} else if (account1.AccountType == beancountAssetsAccountType || account1.AccountType == beancountLiabilitiesAccountType) &&
(account2.AccountType == beancountAssetsAccountType || account2.AccountType == beancountLiabilitiesAccountType) {
var fromAccount, toAccount *beancountAccount
var fromAmount, toAmount int64
var fromCurrency, toCurrency string
if amount1 < 0 {
fromAccount = account1
fromCurrency = splitData1.commodity
fromCurrency = splitData1.Commodity
fromAmount = -amount1
toAccount = account2
toCurrency = splitData2.commodity
toCurrency = splitData2.Commodity
toAmount = amount2
} else if amount2 < 0 {
fromAccount = account2
fromCurrency = splitData2.commodity
fromCurrency = splitData2.Commodity
fromAmount = -amount2
toAccount = account1
toCurrency = splitData1.commodity
toCurrency = splitData1.Commodity
toAmount = amount1
} else {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse transfer transaction, because unexcepted account amounts \"%d\" and \"%d\"", amount1, amount2)
@@ -212,26 +212,26 @@ func (t *beancountTransactionDataRowIterator) parseTransaction(ctx core.Context,
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_TRANSFER))
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = ""
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = fromAccount.name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = fromAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = fromCurrency
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(fromAmount)
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = toAccount.name
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = toAccount.Name
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY] = toCurrency
data[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = utils.FormatAmount(toAmount)
} else {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse transaction, because unexcepted account types \"%d\" and \"%d\"", account1.accountType, account2.accountType)
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse transaction, because unexcepted account types \"%d\" and \"%d\"", account1.AccountType, account2.AccountType)
return nil, errs.ErrThereAreNotSupportedTransactionType
}
} else if len(beancountEntry.postings) <= 1 {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse transaction, because postings count is %d", len(beancountEntry.postings))
} else if len(beancountEntry.Postings) <= 1 {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse transaction, because postings count is %d", len(beancountEntry.Postings))
return nil, errs.ErrInvalidBeancountFile
} else {
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse split transaction, because postings count is %d", len(beancountEntry.postings))
log.Errorf(ctx, "[beancount_transaction_data_table.parseTransaction] cannot parse split transaction, because postings count is %d", len(beancountEntry.Postings))
return nil, errs.ErrNotSupportedSplitTransactions
}
data[datatable.TRANSACTION_DATA_TABLE_TAGS] = strings.Join(beancountEntry.tags, BEANCOUNT_TRANSACTION_TAG_SEPARATOR)
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = beancountEntry.narration
data[datatable.TRANSACTION_DATA_TABLE_TAGS] = strings.Join(beancountEntry.Tags, BEANCOUNT_TRANSACTION_TAG_SEPARATOR)
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = beancountEntry.Narration
return data, nil
}
@@ -242,7 +242,7 @@ func createNewBeancountTransactionDataTable(beancountData *beancountData) (*bean
}
return &beancountTransactionDataTable{
allData: beancountData.transactions,
accountMap: beancountData.accounts,
allData: beancountData.Transactions,
accountMap: beancountData.Accounts,
}, nil
}
+67
View File
@@ -0,0 +1,67 @@
package camt
import "encoding/xml"
type camtCreditDebitIndicator string
const (
CAMT_INDICATOR_CREDIT camtCreditDebitIndicator = "CRDT"
CAMT_INDICATOR_DEBIT camtCreditDebitIndicator = "DBIT"
)
type camt053File struct {
XMLName xml.Name `xml:"Document"`
BankToCustomerStatement *camtBankToCustomerStatement `xml:"BkToCstmrStmt"`
}
type camtBankToCustomerStatement struct {
Statements []*camtStatement `xml:"Stmt"`
}
type camtStatement struct {
Account *camtAccount `xml:"Acct"`
Entries []*camtEntry `xml:"Ntry"`
}
type camtAccount struct {
IBAN string `xml:"Id>IBAN"`
OtherIdentification string `xml:"Id>Othr>Id"`
Currency string `xml:"Ccy"`
}
type camtEntry struct {
Amount *camtAmount `xml:"Amt"`
CreditDebitIndicator camtCreditDebitIndicator `xml:"CdtDbtInd"`
BookingDate *camtDate `xml:"BookgDt"`
EntryDetails *camtEntryDetails `xml:"NtryDtls"`
AdditionalEntryInformation string `xml:"AddtlNtryInf"`
}
type camtAmount struct {
Value string `xml:",chardata"`
Currency string `xml:"Ccy,attr"`
}
type camtDate struct {
Date string `xml:"Dt"`
DateTime string `xml:"DtTm"`
}
type camtEntryDetails struct {
TransactionDetails []*camtTransactionDetails `xml:"TxDtls"`
}
type camtTransactionDetails struct {
AmountDetails *camtAmountDetails `xml:"AmtDtls"`
RemittanceInformation *camtRemittanceInformation `xml:"RmtInf"`
AdditionalTransactionInformation string `xml:"AddtlTxInf"`
}
type camtAmountDetails struct {
InstructedAmount *camtAmount `xml:"InstdAmt>Amt"`
TransactionAmount *camtAmount `xml:"TxAmt>Amt"`
}
type camtRemittanceInformation struct {
Unstructured []string `xml:"Ustrd"`
}
+43
View File
@@ -0,0 +1,43 @@
package camt
import (
"bytes"
"encoding/xml"
"golang.org/x/net/html/charset"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
// camt053FileReader defines the structure of camt.053 file reader
type camt053FileReader struct {
xmlDecoder *xml.Decoder
}
// read returns the imported camt.053 data
// Reference: https://www.iso20022.org/message-set/1196/download
func (r *camt053FileReader) read(ctx core.Context) (*camt053File, error) {
file := &camt053File{}
err := r.xmlDecoder.Decode(&file)
if err != nil {
return nil, err
}
return file, nil
}
func createNewCamt053FileReader(data []byte) (*camt053FileReader, error) {
if len(data) > 5 && data[0] == 0x3C && data[1] == 0x3F && data[2] == 0x78 && data[3] == 0x6D && data[4] == 0x6C { // <?xml
xmlDecoder := xml.NewDecoder(bytes.NewReader(data))
xmlDecoder.CharsetReader = charset.NewReaderLabel
return &camt053FileReader{
xmlDecoder: xmlDecoder,
}, nil
}
return nil, errs.ErrInvalidXmlFile
}
@@ -0,0 +1,314 @@
package camt
import (
"fmt"
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
var camtTransactionSupportedColumns = map[datatable.TransactionDataTableColumn]bool{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: true,
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY: true,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION: true,
}
// camtStatementTransactionDataTable defines the structure of camt statement transaction data table
type camtStatementTransactionDataTable struct {
allStatements []*camtStatement
}
// camtStatementTransactionDataRow defines the structure of camt statement transaction data row
type camtStatementTransactionDataRow struct {
dataTable *camtStatementTransactionDataTable
account *camtAccount
entry *camtEntry
transactionDetails *camtTransactionDetails
finalItems map[datatable.TransactionDataTableColumn]string
}
// camtStatementTransactionDataRowIterator defines the structure of camt statement transaction data row iterator
type camtStatementTransactionDataRowIterator struct {
dataTable *camtStatementTransactionDataTable
currentStatementIndex int
currentEntryIndex int
currentTransactionDetailsIndex int
}
// HasColumn returns whether the transaction data table has specified column
func (t *camtStatementTransactionDataTable) HasColumn(column datatable.TransactionDataTableColumn) bool {
_, exists := camtTransactionSupportedColumns[column]
return exists
}
// TransactionRowCount returns the total count of transaction data row
func (t *camtStatementTransactionDataTable) TransactionRowCount() int {
totalDataRowCount := 0
for i := 0; i < len(t.allStatements); i++ {
statement := t.allStatements[i]
for j := 0; j < len(statement.Entries); j++ {
entry := statement.Entries[j]
if entry.EntryDetails != nil {
totalDataRowCount += len(entry.EntryDetails.TransactionDetails)
} else {
totalDataRowCount++
}
}
}
return totalDataRowCount
}
// TransactionRowIterator returns the iterator of transaction data row
func (t *camtStatementTransactionDataTable) TransactionRowIterator() datatable.TransactionDataRowIterator {
return &camtStatementTransactionDataRowIterator{
dataTable: t,
currentStatementIndex: 0,
currentEntryIndex: 0,
currentTransactionDetailsIndex: -1,
}
}
// IsValid returns whether this row is valid data for importing
func (r *camtStatementTransactionDataRow) IsValid() bool {
return true
}
// GetData returns the data in the specified column type
func (r *camtStatementTransactionDataRow) GetData(column datatable.TransactionDataTableColumn) string {
_, exists := camtTransactionSupportedColumns[column]
if exists {
return r.finalItems[column]
}
return ""
}
// HasNext returns whether the iterator does not reach the end
func (t *camtStatementTransactionDataRowIterator) HasNext() bool {
allStatements := t.dataTable.allStatements
if t.currentStatementIndex >= len(allStatements) {
return false
}
currentStatement := allStatements[t.currentStatementIndex]
if t.currentEntryIndex+1 < len(currentStatement.Entries) {
return true
} else if t.currentEntryIndex < len(currentStatement.Entries) {
currencyEntry := currentStatement.Entries[t.currentEntryIndex]
if currencyEntry.EntryDetails != nil {
if t.currentTransactionDetailsIndex+1 < len(currencyEntry.EntryDetails.TransactionDetails) {
return true
}
} else {
if t.currentTransactionDetailsIndex < 0 {
return true
}
}
}
for i := t.currentStatementIndex + 1; i < len(allStatements); i++ {
statement := allStatements[i]
if len(statement.Entries) < 1 {
continue
}
return true
}
return false
}
// Next returns the next transaction data row
func (t *camtStatementTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
allStatements := t.dataTable.allStatements
for i := t.currentStatementIndex; i < len(allStatements); i++ {
foundNextRow := false
statement := allStatements[i]
for j := t.currentEntryIndex; j < len(statement.Entries); j++ {
if statement.Entries[j].EntryDetails != nil {
if t.currentTransactionDetailsIndex+1 < len(statement.Entries[j].EntryDetails.TransactionDetails) {
t.currentTransactionDetailsIndex++
foundNextRow = true
break
}
} else {
if t.currentTransactionDetailsIndex < 0 {
t.currentTransactionDetailsIndex++
foundNextRow = true
break
}
}
t.currentEntryIndex++
t.currentTransactionDetailsIndex = -1
}
if foundNextRow {
break
}
t.currentStatementIndex++
t.currentEntryIndex = 0
t.currentTransactionDetailsIndex = -1
}
if t.currentStatementIndex >= len(allStatements) {
return nil, nil
}
currentStatement := allStatements[t.currentStatementIndex]
if t.currentEntryIndex >= len(currentStatement.Entries) {
return nil, nil
}
account := currentStatement.Account
entry := currentStatement.Entries[t.currentEntryIndex]
var transactionDetails *camtTransactionDetails
if entry.EntryDetails != nil {
if t.currentTransactionDetailsIndex >= len(entry.EntryDetails.TransactionDetails) {
return nil, nil
} else {
transactionDetails = entry.EntryDetails.TransactionDetails[t.currentTransactionDetailsIndex]
}
} else {
if t.currentTransactionDetailsIndex >= 1 {
return nil, nil
}
}
rowItems, err := t.parseTransaction(ctx, user, account, entry, transactionDetails)
if err != nil {
log.Errorf(ctx, "[camt_statement_transaction_data_table.Next] cannot parsing transaction in entry#%d-transaction_detail#%d (statement#%d), because %s", t.currentEntryIndex, t.currentTransactionDetailsIndex, t.currentStatementIndex, err.Error())
return nil, err
}
return &camtStatementTransactionDataRow{
dataTable: t.dataTable,
account: account,
entry: entry,
transactionDetails: transactionDetails,
finalItems: rowItems,
}, nil
}
func (t *camtStatementTransactionDataRowIterator) parseTransaction(ctx core.Context, user *models.User, account *camtAccount, entry *camtEntry, transactionDetails *camtTransactionDetails) (map[datatable.TransactionDataTableColumn]string, error) {
data := make(map[datatable.TransactionDataTableColumn]string, len(camtTransactionSupportedColumns))
if account == nil {
return nil, errs.ErrMissingAccountData
}
if entry.BookingDate != nil && entry.BookingDate.DateTime != "" {
dateTime, err := utils.ParseFromLongDateTimeWithTimezoneRFC3339Format(entry.BookingDate.DateTime)
if err != nil {
return nil, errs.ErrTransactionTimeInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = utils.FormatUnixTimeToLongDateTime(dateTime.Unix(), dateTime.Location())
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE] = utils.FormatTimezoneOffset(dateTime.Location())
} else if entry.BookingDate != nil && entry.BookingDate.Date != "" {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = fmt.Sprintf("%s 00:00:00", entry.BookingDate.Date)
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE] = datatable.TRANSACTION_DATA_TABLE_TIMEZONE_NOT_AVAILABLE
} else {
return nil, errs.ErrMissingTransactionTime
}
if account.IBAN != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = account.IBAN
} else if account.OtherIdentification != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = account.OtherIdentification
}
if transactionDetails != nil && transactionDetails.AmountDetails != nil && transactionDetails.AmountDetails.TransactionAmount != nil && transactionDetails.AmountDetails.TransactionAmount.Currency != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = transactionDetails.AmountDetails.TransactionAmount.Currency
} else if entry.Amount != nil && entry.Amount.Currency != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = entry.Amount.Currency
} else if account.Currency != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = account.Currency
} else {
return nil, errs.ErrAccountCurrencyInvalid
}
amountValue := ""
if entry.EntryDetails != nil && len(entry.EntryDetails.TransactionDetails) > 1 && transactionDetails != nil { // when there are multiple transaction details in one entry, only use the amount in the transaction details
if transactionDetails.AmountDetails != nil && transactionDetails.AmountDetails.InstructedAmount != nil && transactionDetails.AmountDetails.InstructedAmount.Value != "" {
amountValue = transactionDetails.AmountDetails.InstructedAmount.Value
} else if transactionDetails.AmountDetails != nil && transactionDetails.AmountDetails.TransactionAmount != nil && transactionDetails.AmountDetails.TransactionAmount.Value != "" {
amountValue = transactionDetails.AmountDetails.TransactionAmount.Value
} else {
return nil, errs.ErrAmountInvalid
}
} else if entry.Amount != nil && entry.Amount.Value != "" {
amountValue = entry.Amount.Value
}
if amountValue == "" {
return nil, errs.ErrAmountInvalid
}
amount, err := utils.ParseAmount(amountValue)
if err != nil {
log.Errorf(ctx, "[camt_statement_transaction_data_table.parseTransaction] cannot parsing transaction amount \"%s\", because %s", amountValue, err.Error())
return nil, errs.ErrAmountInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
if entry.CreditDebitIndicator == CAMT_INDICATOR_CREDIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_INCOME))
} else if entry.CreditDebitIndicator == CAMT_INDICATOR_DEBIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE))
} else {
return nil, errs.ErrTransactionTypeInvalid
}
if transactionDetails != nil && transactionDetails.AdditionalTransactionInformation != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = transactionDetails.AdditionalTransactionInformation
} else if transactionDetails != nil && transactionDetails.RemittanceInformation != nil && len(transactionDetails.RemittanceInformation.Unstructured) > 0 {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = strings.Join(transactionDetails.RemittanceInformation.Unstructured, "\n")
} else if entry.AdditionalEntryInformation != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = entry.AdditionalEntryInformation
} else {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = ""
}
return data, nil
}
func createNewCamtStatementTransactionDataTable(file *camt053File) (*camtStatementTransactionDataTable, error) {
if file == nil || file.BankToCustomerStatement == nil || len(file.BankToCustomerStatement.Statements) == 0 {
return nil, errs.ErrNotFoundTransactionDataInFile
}
return &camtStatementTransactionDataTable{
allStatements: file.BankToCustomerStatement.Statements,
}, nil
}
@@ -0,0 +1,48 @@
package camt
import (
"github.com/mayswind/ezbookkeeping/pkg/converters/converter"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
var camtTransactionTypeNameMapping = map[models.TransactionType]string{
models.TRANSACTION_TYPE_INCOME: utils.IntToString(int(models.TRANSACTION_TYPE_INCOME)),
models.TRANSACTION_TYPE_EXPENSE: utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE)),
models.TRANSACTION_TYPE_TRANSFER: utils.IntToString(int(models.TRANSACTION_TYPE_TRANSFER)),
}
// camt053TransactionDataImporter defines the structure of camt.053 file importer for transaction data
type camt053TransactionDataImporter struct {
}
// Initialize a camt.053 transaction data importer singleton instance
var (
Camt053TransactionDataImporter = &camt053TransactionDataImporter{}
)
// ParseImportedData returns the imported data by parsing the camt.053 file transaction data
func (c *camt053TransactionDataImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
camt053DataReader, err := createNewCamt053FileReader(data)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
camt053Data, err := camt053DataReader.read(ctx)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
transactionDataTable, err := createNewCamtStatementTransactionDataTable(camt053Data)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(camtTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -0,0 +1,765 @@
package camt
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
func TestCamt053TransactionDataFileParseImportedData_MinimumValidData(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, allNewAccounts, allNewSubExpenseCategories, allNewSubIncomeCategories, allNewSubTransferCategories, allNewTags, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T01:23:45+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>DBIT</CdtDbtInd>
<Amt Ccy="CNY">0.12</Amt>
</Ntry>
</Stmt>
<Stmt>
<Acct>
<Id>
<Othr>
<Id>456</Id>
</Othr>
</Id>
<Ccy>USD</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T23:59:59+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">1.23</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 3, len(allNewTransactions))
assert.Equal(t, 2, len(allNewAccounts))
assert.Equal(t, 1, len(allNewSubExpenseCategories))
assert.Equal(t, 1, len(allNewSubIncomeCategories))
assert.Equal(t, 0, len(allNewSubTransferCategories))
assert.Equal(t, 0, len(allNewTags))
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, int64(1725125025), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
assert.Equal(t, int64(12345), allNewTransactions[0].Amount)
assert.Equal(t, "123", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "CNY", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, "", allNewTransactions[0].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[1].Type)
assert.Equal(t, int64(1725165296), utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime))
assert.Equal(t, int64(12), allNewTransactions[1].Amount)
assert.Equal(t, "123", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "CNY", allNewTransactions[1].OriginalSourceAccountCurrency)
assert.Equal(t, "", allNewTransactions[1].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[2].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[2].Type)
assert.Equal(t, int64(1725206399), utils.GetUnixTimeFromTransactionTime(allNewTransactions[2].TransactionTime))
assert.Equal(t, int64(123), allNewTransactions[2].Amount)
assert.Equal(t, "456", allNewTransactions[2].OriginalSourceAccountName)
assert.Equal(t, "USD", allNewTransactions[2].OriginalSourceAccountCurrency)
assert.Equal(t, "", allNewTransactions[2].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewAccounts[0].Uid)
assert.Equal(t, "123", allNewAccounts[0].Name)
assert.Equal(t, "CNY", allNewAccounts[0].Currency)
assert.Equal(t, int64(1234567890), allNewAccounts[1].Uid)
assert.Equal(t, "456", allNewAccounts[1].Name)
assert.Equal(t, "USD", allNewAccounts[1].Currency)
assert.Equal(t, int64(1234567890), allNewSubExpenseCategories[0].Uid)
assert.Equal(t, "", allNewSubExpenseCategories[0].Name)
assert.Equal(t, int64(1234567890), allNewSubIncomeCategories[0].Uid)
assert.Equal(t, "", allNewSubIncomeCategories[0].Name)
}
func TestCamt053TransactionDataFileParseImportedData_ParseValidTransactionTime(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<Dt>2024-09-01</Dt>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
<Ntry>
<BookgDt>
<DtTm>2024-09-02T03:04:05Z</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 3, len(allNewTransactions))
assert.Equal(t, int64(1725148800), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
assert.Equal(t, int64(1725165296), utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime))
assert.Equal(t, int64(1725246245), utils.GetUnixTimeFromTransactionTime(allNewTransactions[2].TransactionTime))
}
func TestCamt053TransactionDataFileParseImportedData_ParseInvalidTransactionTime(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024T1</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01 12:34:56</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<Dt>2024/09/01</Dt>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
}
func TestCamt053TransactionDataFileParseImportedData_ParseTransactionValidAmountAndCurrency(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">123.45</Amt>
<NtryDtls>
<TxDtls>
<AmtDtls>
<TxAmt>
<Amt Ccy="USD">100.23</Amt>
</TxAmt>
</AmtDtls>
</TxDtls>
<TxDtls>
<AmtDtls>
<TxAmt>
<Amt Ccy="USD">23.22</Amt>
</TxAmt>
</AmtDtls>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 2, len(allNewTransactions))
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, int64(2322), allNewTransactions[0].Amount)
assert.Equal(t, "USD", allNewTransactions[1].OriginalSourceAccountCurrency)
assert.Equal(t, int64(10023), allNewTransactions[1].Amount)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">123.45</Amt>
<NtryDtls>
<TxDtls>
<AmtDtls>
<InstdAmt>
<Amt Ccy="USD">99.99</Amt>
</InstdAmt>
<TxAmt>
<Amt Ccy="USD">100.23</Amt>
</TxAmt>
</AmtDtls>
</TxDtls>
<TxDtls>
<AmtDtls>
<InstdAmt>
<Amt Ccy="USD">23.46</Amt>
</InstdAmt>
<TxAmt>
<Amt Ccy="USD">23.22</Amt>
</TxAmt>
</AmtDtls>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 2, len(allNewTransactions))
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, int64(2346), allNewTransactions[0].Amount)
assert.Equal(t, "USD", allNewTransactions[1].OriginalSourceAccountCurrency)
assert.Equal(t, int64(9999), allNewTransactions[1].Amount)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">123.45</Amt>
<NtryDtls>
<TxDtls>
<AmtDtls>
<TxAmt>
<Amt Ccy="USD">123.45</Amt>
</TxAmt>
</AmtDtls>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, int64(12345), allNewTransactions[0].Amount)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt>123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "CNY", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, int64(12345), allNewTransactions[0].Amount)
}
func TestCamt053TransactionDataFileParseImportedData_ParseTransactionInvalidAmountAndCurrency(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt>123 45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">123.45</Amt>
<NtryDtls>
<TxDtls>
<AmtDtls>
</AmtDtls>
</TxDtls>
<TxDtls>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="USD">123.45</Amt>
<NtryDtls>
<TxDtls>
</TxDtls>
<TxDtls>
<AmtDtls>
</AmtDtls>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
}
func TestCamt053TransactionDataFileParseImportedData_ParseDescription(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
<AddtlNtryInf>Test Entry</AddtlNtryInf>
<NtryDtls>
<TxDtls>
<AddtlTxInf>Test Transaction</AddtlTxInf>
<RmtInf>
<Ustrd>Test Line 1</Ustrd>
<Ustrd>Test Line 2</Ustrd>
</RmtInf>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "Test Transaction", allNewTransactions[0].Comment)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
<AddtlNtryInf>Test Entry</AddtlNtryInf>
<NtryDtls>
<TxDtls>
<RmtInf>
<Ustrd>Test Line 1</Ustrd>
<Ustrd>Test Line 2</Ustrd>
</RmtInf>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "Test Line 1\nTest Line 2", allNewTransactions[0].Comment)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
<AddtlNtryInf>Test Entry</AddtlNtryInf>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "Test Entry", allNewTransactions[0].Comment)
}
func TestCamt053TransactionDataFileParseImportedData_MissingAccountNode(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingAccountData.Message)
}
func TestCamt053TransactionDataFileParseImportedData_MissingTransactionRequiredNode(t *testing.T) {
converter := Camt053TransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingTransactionTime.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<Amt Ccy="CNY">123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTypeInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
<Ccy>CNY</Ccy>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<Stmt>
<Acct>
<Id>
<IBAN>123</IBAN>
</Id>
</Acct>
<Ntry>
<BookgDt>
<DtTm>2024-09-01T12:34:56+08:00</DtTm>
</BookgDt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Amt>123.45</Amt>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
}
@@ -13,17 +13,25 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/validators"
)
type TransactionGeoLocationOrder string
const (
TRANSACTION_GEO_LOCATION_ORDER_LONGITUDE_LATITUDE TransactionGeoLocationOrder = "lonlat" // longitude first, then latitude
TRANSACTION_GEO_LOCATION_ORDER_LATITUDE_LONGITUDE TransactionGeoLocationOrder = "latlon" // latitude first, then longitude
)
// DataTableTransactionDataImporter defines the structure of plain text data table importer for transaction data
type DataTableTransactionDataImporter struct {
transactionTypeMapping map[string]models.TransactionType
geoLocationSeparator string
geoLocationOrder TransactionGeoLocationOrder
transactionTagSeparator string
}
// ParseImportedData returns the imported transaction data
func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, user *models.User, dataTable datatable.TransactionDataTable, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
if dataTable.TransactionRowCount() < 1 {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse import data for user \"uid:%d\", because data table row count is less 1", user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse import data for user \"uid:%d\", because data table row count is less 1", user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrNotFoundTransactionDataInFile
}
@@ -39,7 +47,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
!dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME) ||
!dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_AMOUNT) ||
!dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME) {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse import data for user \"uid:%d\", because missing essential columns in header row", user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse import data for user \"uid:%d\", because missing essential columns in header row", user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrMissingRequiredFieldInHeaderRow
}
@@ -78,7 +86,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
dataRow, err := dataRowIterator.Next(ctx, user)
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse data row \"index:%d\" for user \"uid:%d\", because %s", dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse data row \"index:%d\" for user \"uid:%d\", because %s", dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, err
}
@@ -88,11 +96,12 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
timezoneOffset := defaultTimezoneOffset
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE) {
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE) &&
dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE) != datatable.TRANSACTION_DATA_TABLE_TIMEZONE_NOT_AVAILABLE {
transactionTimezone, err := utils.ParseFromTimezoneOffset(dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE))
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse time zone \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse time zone \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrTransactionTimeZoneInvalid
}
@@ -102,14 +111,14 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
transactionTime, err := utils.ParseFromLongDateTime(dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME), timezoneOffset)
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse time \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse time \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrTransactionTimeInvalid
}
transactionDbType, err := c.getTransactionDbType(nameDbTypeMap, dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE))
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse transaction type \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse transaction type \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.Or(err, errs.ErrTransactionTypeInvalid)
}
@@ -121,7 +130,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
transactionCategoryType, err := c.getTransactionCategoryType(transactionDbType)
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse transaction category type in data row \"index:%d\" for user \"uid:%d\", because %s", dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse transaction category type in data row \"index:%d\" for user \"uid:%d\", because %s", dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.Or(err, errs.ErrTransactionTypeInvalid)
}
@@ -179,11 +188,11 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
accountName := dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME)
accountCurrency := user.DefaultCurrency
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) {
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) && dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) != "" {
accountCurrency = dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY)
if _, ok := validators.AllCurrencyNames[accountCurrency]; !ok {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] account currency \"%s\" is not supported in data row \"index:%d\" for user \"uid:%d\"", accountCurrency, dataRowIndex, user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] account currency \"%s\" is not supported in data row \"index:%d\" for user \"uid:%d\"", accountCurrency, dataRowIndex, user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrAccountCurrencyInvalid
}
}
@@ -196,9 +205,9 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
accountMap[accountName] = account
}
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) {
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) && dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY) != "" {
if account.Name != "" && account.Currency != accountCurrency {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] currency \"%s\" in data row \"index:%d\" not equals currency \"%s\" of the account for user \"uid:%d\"", accountCurrency, dataRowIndex, account.Currency, user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] currency \"%s\" in data row \"index:%d\" not equals currency \"%s\" of the account for user \"uid:%d\"", accountCurrency, dataRowIndex, account.Currency, user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrAccountCurrencyInvalid
}
} else if exists {
@@ -208,7 +217,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
amount, err := utils.ParseAmount(dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_AMOUNT))
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse acmount \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_AMOUNT), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse acmount \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_AMOUNT), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrAmountInvalid
}
@@ -221,11 +230,11 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
account2Name = dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME)
account2Currency = user.DefaultCurrency
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) {
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) && dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) != "" {
account2Currency = dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY)
if _, ok := validators.AllCurrencyNames[account2Currency]; !ok {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] account2 currency \"%s\" is not supported in data row \"index:%d\" for user \"uid:%d\"", account2Currency, dataRowIndex, user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] account2 currency \"%s\" is not supported in data row \"index:%d\" for user \"uid:%d\"", account2Currency, dataRowIndex, user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrAccountCurrencyInvalid
}
}
@@ -238,9 +247,9 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
accountMap[account2Name] = account2
}
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) {
if dataTable.HasColumn(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) && dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY) != "" {
if account2.Name != "" && account2.Currency != account2Currency {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] currency \"%s\" in data row \"index:%d\" not equals currency \"%s\" of the account2 for user \"uid:%d\"", account2Currency, dataRowIndex, account2.Currency, user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] currency \"%s\" in data row \"index:%d\" not equals currency \"%s\" of the account2 for user \"uid:%d\"", account2Currency, dataRowIndex, account2.Currency, user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrAccountCurrencyInvalid
}
} else if exists {
@@ -253,7 +262,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
relatedAccountAmount, err = utils.ParseAmount(dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT))
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse acmount2 \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse acmount2 \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrAmountInvalid
}
} else if transactionDbType == models.TRANSACTION_DB_TYPE_TRANSFER_OUT {
@@ -268,19 +277,27 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
geoLocationItems := strings.Split(dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION), c.geoLocationSeparator)
if len(geoLocationItems) == 2 {
geoLongitude, err = utils.StringToFloat64(geoLocationItems[0])
geoLocationFirstItem, err := utils.StringToFloat64(geoLocationItems[0])
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse geographic location \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse geographic location \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrGeographicLocationInvalid
}
geoLatitude, err = utils.StringToFloat64(geoLocationItems[1])
geoLocationSecondItem, err := utils.StringToFloat64(geoLocationItems[1])
if err != nil {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] cannot parse geographic location \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION), dataRowIndex, user.Uid, err.Error())
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] cannot parse geographic location \"%s\" in data row \"index:%d\" for user \"uid:%d\", because %s", dataRow.GetData(datatable.TRANSACTION_DATA_TABLE_GEOGRAPHIC_LOCATION), dataRowIndex, user.Uid, err.Error())
return nil, nil, nil, nil, nil, nil, errs.ErrGeographicLocationInvalid
}
if c.geoLocationOrder == TRANSACTION_GEO_LOCATION_ORDER_LONGITUDE_LATITUDE {
geoLongitude = geoLocationFirstItem
geoLatitude = geoLocationSecondItem
} else if c.geoLocationOrder == TRANSACTION_GEO_LOCATION_ORDER_LATITUDE_LONGITUDE {
geoLatitude = geoLocationFirstItem
geoLongitude = geoLocationSecondItem
}
}
}
@@ -355,7 +372,7 @@ func (c *DataTableTransactionDataImporter) ParseImportedData(ctx core.Context, u
}
if len(allNewTransactions) < 1 {
log.Errorf(ctx, "[data_table_transaction_data_exporter.ParseImportedData] no transaction data parsed for \"uid:%d\"", user.Uid)
log.Errorf(ctx, "[data_table_transaction_data_importer.ParseImportedData] no transaction data parsed for \"uid:%d\"", user.Uid)
return nil, nil, nil, nil, nil, nil, errs.ErrNotFoundTransactionDataInFile
}
@@ -466,10 +483,11 @@ func (c *DataTableTransactionDataImporter) createNewTransactionTagModel(uid int6
}
// CreateNewImporterWithTypeNameMapping returns a new data table transaction data importer according to the specified arguments
func CreateNewImporterWithTypeNameMapping(transactionTypeMapping map[models.TransactionType]string, geoLocationSeparator string, transactionTagSeparator string) *DataTableTransactionDataImporter {
func CreateNewImporterWithTypeNameMapping(transactionTypeMapping map[models.TransactionType]string, geoLocationSeparator string, geoLocationOrder TransactionGeoLocationOrder, transactionTagSeparator string) *DataTableTransactionDataImporter {
return &DataTableTransactionDataImporter{
transactionTypeMapping: buildTransactionNameTypeMap(transactionTypeMapping),
geoLocationSeparator: geoLocationSeparator,
geoLocationOrder: geoLocationOrder,
transactionTagSeparator: transactionTagSeparator,
}
}
@@ -0,0 +1,138 @@
package csv
import (
"encoding/csv"
"fmt"
"io"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
)
// CsvFileBasicDataTable defines the structure of csv data table
type CsvFileBasicDataTable struct {
allLines [][]string
}
// CsvFileBasicDataTableRow defines the structure of csv data table row
type CsvFileBasicDataTableRow struct {
dataTable *CsvFileBasicDataTable
allItems []string
}
// CsvFileBasicDataTableRowIterator defines the structure of csv data table row iterator
type CsvFileBasicDataTableRowIterator struct {
dataTable *CsvFileBasicDataTable
currentIndex int
}
// DataRowCount returns the total count of data row
func (t *CsvFileBasicDataTable) DataRowCount() int {
if len(t.allLines) < 1 {
return 0
}
return len(t.allLines) - 1
}
// HeaderColumnNames returns the header column name list
func (t *CsvFileBasicDataTable) HeaderColumnNames() []string {
if len(t.allLines) < 1 {
return nil
}
return t.allLines[0]
}
// DataRowIterator returns the iterator of data row
func (t *CsvFileBasicDataTable) DataRowIterator() datatable.BasicDataTableRowIterator {
return &CsvFileBasicDataTableRowIterator{
dataTable: t,
currentIndex: 0,
}
}
// ColumnCount returns the total count of column in this data row
func (r *CsvFileBasicDataTableRow) ColumnCount() int {
return len(r.allItems)
}
// GetData returns the data in the specified column index
func (r *CsvFileBasicDataTableRow) GetData(columnIndex int) string {
if columnIndex >= len(r.allItems) {
return ""
}
return r.allItems[columnIndex]
}
// HasNext returns whether the iterator does not reach the end
func (t *CsvFileBasicDataTableRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allLines)
}
// CurrentRowId returns current index
func (t *CsvFileBasicDataTableRowIterator) CurrentRowId() string {
return fmt.Sprintf("line#%d", t.currentIndex)
}
// Next returns the next basic data row
func (t *CsvFileBasicDataTableRowIterator) Next() datatable.BasicDataTableRow {
if t.currentIndex+1 >= len(t.dataTable.allLines) {
return nil
}
t.currentIndex++
rowItems := t.dataTable.allLines[t.currentIndex]
return &CsvFileBasicDataTableRow{
dataTable: t.dataTable,
allItems: rowItems,
}
}
// CreateNewCsvBasicDataTable returns comma separated values data table by io readers
func CreateNewCsvBasicDataTable(ctx core.Context, reader io.Reader) (datatable.BasicDataTable, error) {
return createNewCsvFileBasicDataTable(ctx, reader, ',')
}
// CreateNewCustomCsvBasicDataTable returns character separated values data table by io readers
func CreateNewCustomCsvBasicDataTable(allLines [][]string) datatable.BasicDataTable {
return &CsvFileBasicDataTable{
allLines: allLines,
}
}
func createNewCsvFileBasicDataTable(ctx core.Context, reader io.Reader, separator rune) (*CsvFileBasicDataTable, error) {
csvReader := csv.NewReader(reader)
csvReader.Comma = separator
csvReader.FieldsPerRecord = -1
allLines := make([][]string, 0)
for {
items, err := csvReader.Read()
if err == io.EOF {
break
}
if err != nil {
log.Errorf(ctx, "[csv_file_basic_data_table.createNewCsvFileDataTable] cannot parse csv data, because %s", err.Error())
return nil, errs.ErrInvalidCSVFile
}
if len(items) == 1 && items[0] == "" {
continue
}
allLines = append(allLines, items)
}
return &CsvFileBasicDataTable{
allLines: allLines,
}, nil
}
@@ -9,8 +9,8 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/core"
)
func TestCsvFileImportedDataTableDataRowCount(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableDataRowCount(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -19,22 +19,22 @@ func TestCsvFileImportedDataTableDataRowCount(t *testing.T) {
assert.Equal(t, 2, datatable.DataRowCount())
}
func TestCsvFileImportedDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
})
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestCsvFileImportedDataTableDataRowCount_EmptyContent(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{})
func TestCsvFileBasicDataTableDataRowCount_EmptyContent(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{})
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestCsvFileImportedDataTableHeaderColumnNames(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableHeaderColumnNames(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -43,14 +43,14 @@ func TestCsvFileImportedDataTableHeaderColumnNames(t *testing.T) {
assert.EqualValues(t, []string{"A1", "B1", "C1"}, datatable.HeaderColumnNames())
}
func TestCsvFileImportedDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{})
func TestCsvFileBasicDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{})
assert.Nil(t, datatable.HeaderColumnNames())
}
func TestCsvFileImportedDataRowIterator(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableRowIterator(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -76,8 +76,8 @@ func TestCsvFileImportedDataRowIterator(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestCsvFileImportedDataRowColumnCount(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableRowColumnCount(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -92,8 +92,8 @@ func TestCsvFileImportedDataRowColumnCount(t *testing.T) {
assert.EqualValues(t, 3, row2.ColumnCount())
}
func TestCsvFileImportedDataRowGetData(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableRowGetData(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -112,8 +112,8 @@ func TestCsvFileImportedDataRowGetData(t *testing.T) {
assert.Equal(t, "C3", row2.GetData(2))
}
func TestCsvFileImportedDataRowGetData_GetNotExistedColumnData(t *testing.T) {
datatable := CreateNewCustomCsvImportedDataTable([][]string{
func TestCsvFileBasicDataTableRowGetData_GetNotExistedColumnData(t *testing.T) {
datatable := CreateNewCustomCsvBasicDataTable([][]string{
{"A1", "B1", "C1"},
{"A2", "B2", "C2"},
{"A3", "B3", "C3"},
@@ -125,12 +125,12 @@ func TestCsvFileImportedDataRowGetData_GetNotExistedColumnData(t *testing.T) {
assert.Equal(t, "", row1.GetData(3))
}
func TestCreateNewCsvImportedDataTable(t *testing.T) {
func TestCreateNewCsvBasicDataTable(t *testing.T) {
context := core.NewNullContext()
reader := bytes.NewReader([]byte("A1,B1,C1\n" +
"A2,B2,C2\n" +
"A3,B3,C3\n"))
datatable, err := CreateNewCsvImportedDataTable(context, reader)
datatable, err := CreateNewCsvBasicDataTable(context, reader)
assert.Nil(t, err)
assert.Equal(t, 2, datatable.DataRowCount())
@@ -153,14 +153,14 @@ func TestCreateNewCsvImportedDataTable(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestCreateNewCsvImportedDataTable_SkipBlankLine(t *testing.T) {
func TestCreateNewCsvBasicDataTable_SkipBlankLine(t *testing.T) {
context := core.NewNullContext()
reader := bytes.NewReader([]byte("\n" +
"A1,B1,C1\n" +
"A2,B2,C2\n" +
"\n" +
"A3,B3,C3\n"))
datatable, err := CreateNewCsvImportedDataTable(context, reader)
datatable, err := CreateNewCsvBasicDataTable(context, reader)
assert.Nil(t, err)
assert.Equal(t, 2, datatable.DataRowCount())
@@ -1,138 +0,0 @@
package csv
import (
"encoding/csv"
"fmt"
"io"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
)
// CsvFileImportedDataTable defines the structure of csv data table
type CsvFileImportedDataTable struct {
allLines [][]string
}
// CsvFileImportedDataRow defines the structure of csv data table row
type CsvFileImportedDataRow struct {
dataTable *CsvFileImportedDataTable
allItems []string
}
// CsvFileImportedDataRowIterator defines the structure of csv data table row iterator
type CsvFileImportedDataRowIterator struct {
dataTable *CsvFileImportedDataTable
currentIndex int
}
// DataRowCount returns the total count of data row
func (t *CsvFileImportedDataTable) DataRowCount() int {
if len(t.allLines) < 1 {
return 0
}
return len(t.allLines) - 1
}
// HeaderColumnNames returns the header column name list
func (t *CsvFileImportedDataTable) HeaderColumnNames() []string {
if len(t.allLines) < 1 {
return nil
}
return t.allLines[0]
}
// DataRowIterator returns the iterator of data row
func (t *CsvFileImportedDataTable) DataRowIterator() datatable.ImportedDataRowIterator {
return &CsvFileImportedDataRowIterator{
dataTable: t,
currentIndex: 0,
}
}
// ColumnCount returns the total count of column in this data row
func (r *CsvFileImportedDataRow) ColumnCount() int {
return len(r.allItems)
}
// GetData returns the data in the specified column index
func (r *CsvFileImportedDataRow) GetData(columnIndex int) string {
if columnIndex >= len(r.allItems) {
return ""
}
return r.allItems[columnIndex]
}
// HasNext returns whether the iterator does not reach the end
func (t *CsvFileImportedDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allLines)
}
// CurrentRowId returns current index
func (t *CsvFileImportedDataRowIterator) CurrentRowId() string {
return fmt.Sprintf("line#%d", t.currentIndex)
}
// Next returns the next imported data row
func (t *CsvFileImportedDataRowIterator) Next() datatable.ImportedDataRow {
if t.currentIndex+1 >= len(t.dataTable.allLines) {
return nil
}
t.currentIndex++
rowItems := t.dataTable.allLines[t.currentIndex]
return &CsvFileImportedDataRow{
dataTable: t.dataTable,
allItems: rowItems,
}
}
// CreateNewCsvImportedDataTable returns comma separated values data table by io readers
func CreateNewCsvImportedDataTable(ctx core.Context, reader io.Reader) (*CsvFileImportedDataTable, error) {
return createNewCsvFileDataTable(ctx, reader, ',')
}
// CreateNewCustomCsvImportedDataTable returns character separated values data table by io readers
func CreateNewCustomCsvImportedDataTable(allLines [][]string) *CsvFileImportedDataTable {
return &CsvFileImportedDataTable{
allLines: allLines,
}
}
func createNewCsvFileDataTable(ctx core.Context, reader io.Reader, separator rune) (*CsvFileImportedDataTable, error) {
csvReader := csv.NewReader(reader)
csvReader.Comma = separator
csvReader.FieldsPerRecord = -1
allLines := make([][]string, 0)
for {
items, err := csvReader.Read()
if err == io.EOF {
break
}
if err != nil {
log.Errorf(ctx, "[csv_file_imported_data_table.createNewCsvFileDataTable] cannot parse csv data, because %s", err.Error())
return nil, errs.ErrInvalidCSVFile
}
if len(items) == 1 && items[0] == "" {
continue
}
allLines = append(allLines, items)
}
return &CsvFileImportedDataTable{
allLines: allLines,
}, nil
}
@@ -1,7 +1,7 @@
package datatable
// ImportedDataTable defines the structure of imported data table
type ImportedDataTable interface {
// BasicDataTable defines the structure of basic data table
type BasicDataTable interface {
// DataRowCount returns the total count of data row
DataRowCount() int
@@ -9,11 +9,11 @@ type ImportedDataTable interface {
HeaderColumnNames() []string
// DataRowIterator returns the iterator of data row
DataRowIterator() ImportedDataRowIterator
DataRowIterator() BasicDataTableRowIterator
}
// ImportedDataRow defines the structure of imported data row
type ImportedDataRow interface {
// BasicDataTableRow defines the structure of basic data row
type BasicDataTableRow interface {
// ColumnCount returns the total count of column in this data row
ColumnCount() int
@@ -21,14 +21,14 @@ type ImportedDataRow interface {
GetData(columnIndex int) string
}
// ImportedDataRowIterator defines the structure of imported data row iterator
type ImportedDataRowIterator interface {
// BasicDataTableRowIterator defines the structure of basic data row iterator
type BasicDataTableRowIterator interface {
// HasNext returns whether the iterator does not reach the end
HasNext() bool
// CurrentRowId returns current row id
CurrentRowId() string
// Next returns the next imported data row
Next() ImportedDataRow
// Next returns the next basic data row
Next() BasicDataTableRow
}
@@ -0,0 +1,107 @@
package datatable
// basicDataTableToCommonDataTableWrapper defines the structure of basic data table to common data table wrapper
type basicDataTableToCommonDataTableWrapper struct {
innerDataTable BasicDataTable
dataColumnIndexes map[string]int
}
// basicDataTableToCommonDataTableWrapperRow defines the data row structure of basic data table to common data table wrapper
type basicDataTableToCommonDataTableWrapperRow struct {
rowData map[string]string
}
// basicDataTableToCommonDataTableWrapperRowIterator defines the data row iterator structure of basic data table to common data table wrapper
type basicDataTableToCommonDataTableWrapperRowIterator struct {
commonDataTable *basicDataTableToCommonDataTableWrapper
innerIterator BasicDataTableRowIterator
}
// HeaderColumnCount returns the total count of column in header row
func (t *basicDataTableToCommonDataTableWrapper) HeaderColumnCount() int {
return len(t.innerDataTable.HeaderColumnNames())
}
// HasColumn returns whether the data table has specified column name
func (t *basicDataTableToCommonDataTableWrapper) HasColumn(columnName string) bool {
index, exists := t.dataColumnIndexes[columnName]
return exists && index >= 0
}
// DataRowCount returns the total count of common data row
func (t *basicDataTableToCommonDataTableWrapper) DataRowCount() int {
return t.innerDataTable.DataRowCount()
}
// DataRowIterator returns the iterator of common data row
func (t *basicDataTableToCommonDataTableWrapper) DataRowIterator() CommonDataTableRowIterator {
return &basicDataTableToCommonDataTableWrapperRowIterator{
commonDataTable: t,
innerIterator: t.innerDataTable.DataRowIterator(),
}
}
// HasData returns whether the common data row has specified column data
func (r *basicDataTableToCommonDataTableWrapperRow) HasData(columnName string) bool {
_, exists := r.rowData[columnName]
return exists
}
// ColumnCount returns the total count of column in this data row
func (r *basicDataTableToCommonDataTableWrapperRow) ColumnCount() int {
return len(r.rowData)
}
// GetData returns the data in the specified column name
func (r *basicDataTableToCommonDataTableWrapperRow) GetData(columnName string) string {
return r.rowData[columnName]
}
// HasNext returns whether the iterator does not reach the end
func (t *basicDataTableToCommonDataTableWrapperRowIterator) HasNext() bool {
return t.innerIterator.HasNext()
}
// CurrentRowId returns current row id
func (t *basicDataTableToCommonDataTableWrapperRowIterator) CurrentRowId() string {
return t.innerIterator.CurrentRowId()
}
// Next returns the next common data row
func (t *basicDataTableToCommonDataTableWrapperRowIterator) Next() CommonDataTableRow {
basicDataRow := t.innerIterator.Next()
if basicDataRow == nil {
return nil
}
rowData := make(map[string]string, len(t.commonDataTable.dataColumnIndexes))
for column, columnIndex := range t.commonDataTable.dataColumnIndexes {
if columnIndex < 0 || columnIndex >= basicDataRow.ColumnCount() {
continue
}
value := basicDataRow.GetData(columnIndex)
rowData[column] = value
}
return &basicDataTableToCommonDataTableWrapperRow{
rowData: rowData,
}
}
// CreateNewCommonDataTableFromBasicDataTable returns common data table from basic data table
func CreateNewCommonDataTableFromBasicDataTable(dataTable BasicDataTable) CommonDataTable {
headerLineItems := dataTable.HeaderColumnNames()
dataColumnIndexes := make(map[string]int, len(headerLineItems))
for i := 0; i < len(headerLineItems); i++ {
dataColumnIndexes[headerLineItems[i]] = i
}
return &basicDataTableToCommonDataTableWrapper{
innerDataTable: dataTable,
dataColumnIndexes: dataColumnIndexes,
}
}
@@ -7,30 +7,30 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/models"
)
// ImportedTransactionDataTable defines the structure of imported transaction data table
type ImportedTransactionDataTable struct {
innerDataTable ImportedDataTable
// basicDataTableToTransactionDataTableWrapper defines the structure of basic data table to transaction data table wrapper
type basicDataTableToTransactionDataTableWrapper struct {
innerDataTable BasicDataTable
dataColumnMapping map[TransactionDataTableColumn]string
dataColumnIndexes map[TransactionDataTableColumn]int
rowParser TransactionDataRowParser
addedColumns map[TransactionDataTableColumn]bool
}
// ImportedTransactionDataRow defines the structure of imported transaction data row
type ImportedTransactionDataRow struct {
transactionDataTable *ImportedTransactionDataTable
// basicDataTableToTransactionDataTableWrapperRow defines the data row structure of basic data table to transaction data table wrapper
type basicDataTableToTransactionDataTableWrapperRow struct {
transactionDataTable *basicDataTableToTransactionDataTableWrapper
rowData map[TransactionDataTableColumn]string
rowDataValid bool
}
// ImportedTransactionDataRowIterator defines the structure of imported transaction data row iterator
type ImportedTransactionDataRowIterator struct {
transactionDataTable *ImportedTransactionDataTable
innerIterator ImportedDataRowIterator
// basicDataTableToTransactionDataTableWrapperRowIterator defines the data row iterator structure of basic data table to transaction data table wrapper
type basicDataTableToTransactionDataTableWrapperRowIterator struct {
transactionDataTable *basicDataTableToTransactionDataTableWrapper
innerIterator BasicDataTableRowIterator
}
// HasColumn returns whether the data table has specified column
func (t *ImportedTransactionDataTable) HasColumn(column TransactionDataTableColumn) bool {
func (t *basicDataTableToTransactionDataTableWrapper) HasColumn(column TransactionDataTableColumn) bool {
index, exists := t.dataColumnIndexes[column]
if exists && index >= 0 {
@@ -49,25 +49,25 @@ func (t *ImportedTransactionDataTable) HasColumn(column TransactionDataTableColu
}
// TransactionRowCount returns the total count of transaction data row
func (t *ImportedTransactionDataTable) TransactionRowCount() int {
func (t *basicDataTableToTransactionDataTableWrapper) TransactionRowCount() int {
return t.innerDataTable.DataRowCount()
}
// TransactionRowIterator returns the iterator of transaction data row
func (t *ImportedTransactionDataTable) TransactionRowIterator() TransactionDataRowIterator {
return &ImportedTransactionDataRowIterator{
func (t *basicDataTableToTransactionDataTableWrapper) TransactionRowIterator() TransactionDataRowIterator {
return &basicDataTableToTransactionDataTableWrapperRowIterator{
transactionDataTable: t,
innerIterator: t.innerDataTable.DataRowIterator(),
}
}
// IsValid returns whether this row is valid data for importing
func (r *ImportedTransactionDataRow) IsValid() bool {
func (r *basicDataTableToTransactionDataTableWrapperRow) IsValid() bool {
return r.rowDataValid
}
// GetData returns the data in the specified column type
func (r *ImportedTransactionDataRow) GetData(column TransactionDataTableColumn) string {
func (r *basicDataTableToTransactionDataTableWrapperRow) GetData(column TransactionDataTableColumn) string {
if !r.rowDataValid {
return ""
}
@@ -90,28 +90,28 @@ func (r *ImportedTransactionDataRow) GetData(column TransactionDataTableColumn)
}
// HasNext returns whether the iterator does not reach the end
func (t *ImportedTransactionDataRowIterator) HasNext() bool {
func (t *basicDataTableToTransactionDataTableWrapperRowIterator) HasNext() bool {
return t.innerIterator.HasNext()
}
// Next returns the next transaction data row
func (t *ImportedTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow TransactionDataRow, err error) {
importedRow := t.innerIterator.Next()
func (t *basicDataTableToTransactionDataTableWrapperRowIterator) Next(ctx core.Context, user *models.User) (daraRow TransactionDataRow, err error) {
basicDataRow := t.innerIterator.Next()
if importedRow == nil {
if basicDataRow == nil {
return nil, nil
}
if importedRow.ColumnCount() == 1 && importedRow.GetData(0) == "" {
return &ImportedTransactionDataRow{
if basicDataRow.ColumnCount() == 1 && basicDataRow.GetData(0) == "" {
return &basicDataTableToTransactionDataTableWrapperRow{
transactionDataTable: t.transactionDataTable,
rowData: nil,
rowDataValid: false,
}, nil
}
if importedRow.ColumnCount() < len(t.transactionDataTable.dataColumnIndexes) {
log.Errorf(ctx, "[imported_transaction_data_table.Next] cannot parse data row, because may missing some columns (column count %d in data row is less than header column count %d)", importedRow.ColumnCount(), len(t.transactionDataTable.dataColumnIndexes))
if basicDataRow.ColumnCount() < len(t.transactionDataTable.dataColumnIndexes) {
log.Errorf(ctx, "[basic_data_table_to_transaction_data_table_wrapper.Next] cannot parse data row, because may missing some columns (column count %d in data row is less than header column count %d)", basicDataRow.ColumnCount(), len(t.transactionDataTable.dataColumnIndexes))
return nil, errs.ErrFewerFieldsInDataRowThanInHeaderRow
}
@@ -119,11 +119,11 @@ func (t *ImportedTransactionDataRowIterator) Next(ctx core.Context, user *models
rowDataValid := true
for column, columnIndex := range t.transactionDataTable.dataColumnIndexes {
if columnIndex < 0 || columnIndex >= importedRow.ColumnCount() {
if columnIndex < 0 || columnIndex >= basicDataRow.ColumnCount() {
continue
}
value := importedRow.GetData(columnIndex)
value := basicDataRow.GetData(columnIndex)
rowData[column] = value
}
@@ -131,25 +131,25 @@ func (t *ImportedTransactionDataRowIterator) Next(ctx core.Context, user *models
rowData, rowDataValid, err = t.transactionDataTable.rowParser.Parse(rowData)
if err != nil {
log.Errorf(ctx, "[imported_transaction_data_table.Next] cannot parse data row, because %s", err.Error())
log.Errorf(ctx, "[basic_data_table_to_transaction_data_table_wrapper.Next] cannot parse data row, because %s", err.Error())
return nil, err
}
}
return &ImportedTransactionDataRow{
return &basicDataTableToTransactionDataTableWrapperRow{
transactionDataTable: t.transactionDataTable,
rowData: rowData,
rowDataValid: rowDataValid,
}, nil
}
// CreateNewImportedTransactionDataTable returns transaction data table from imported data table
func CreateNewImportedTransactionDataTable(dataTable ImportedDataTable, dataColumnMapping map[TransactionDataTableColumn]string) *ImportedTransactionDataTable {
return CreateNewImportedTransactionDataTableWithRowParser(dataTable, dataColumnMapping, nil)
// CreateNewTransactionDataTableFromBasicDataTable returns transaction data table from basic data table
func CreateNewTransactionDataTableFromBasicDataTable(dataTable BasicDataTable, dataColumnMapping map[TransactionDataTableColumn]string) TransactionDataTable {
return CreateNewTransactionDataTableFromBasicDataTableWithRowParser(dataTable, dataColumnMapping, nil)
}
// CreateNewImportedTransactionDataTableWithRowParser returns transaction data table from imported data table
func CreateNewImportedTransactionDataTableWithRowParser(dataTable ImportedDataTable, dataColumnMapping map[TransactionDataTableColumn]string, rowParser TransactionDataRowParser) *ImportedTransactionDataTable {
// CreateNewTransactionDataTableFromBasicDataTableWithRowParser returns transaction data table from basic data table
func CreateNewTransactionDataTableFromBasicDataTableWithRowParser(dataTable BasicDataTable, dataColumnMapping map[TransactionDataTableColumn]string, rowParser TransactionDataRowParser) TransactionDataTable {
headerLineItems := dataTable.HeaderColumnNames()
headerItemMap := make(map[string]int, len(headerLineItems))
@@ -178,7 +178,7 @@ func CreateNewImportedTransactionDataTableWithRowParser(dataTable ImportedDataTa
}
}
return &ImportedTransactionDataTable{
return &basicDataTableToTransactionDataTableWrapper{
innerDataTable: dataTable,
dataColumnMapping: dataColumnMapping,
dataColumnIndexes: dataColumnIndexes,
@@ -12,11 +12,11 @@ type CommonDataTable interface {
DataRowCount() int
// DataRowIterator returns the iterator of common data row
DataRowIterator() CommonDataRowIterator
DataRowIterator() CommonDataTableRowIterator
}
// CommonDataRow defines the structure of common data row
type CommonDataRow interface {
// CommonDataTableRow defines the structure of common data row
type CommonDataTableRow interface {
// ColumnCount returns the total count of column in this data row
ColumnCount() int
@@ -27,8 +27,8 @@ type CommonDataRow interface {
GetData(columnName string) string
}
// CommonDataRowIterator defines the structure of common data row iterator
type CommonDataRowIterator interface {
// CommonDataTableRowIterator defines the structure of common data row iterator
type CommonDataTableRowIterator interface {
// HasNext returns whether the iterator does not reach the end
HasNext() bool
@@ -36,5 +36,5 @@ type CommonDataRowIterator interface {
CurrentRowId() string
// Next returns the next common data row
Next() CommonDataRow
Next() CommonDataTableRow
}
@@ -0,0 +1,109 @@
package datatable
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
// CommonTransactionDataRowParser defines the structure of common transaction data row parser
type CommonTransactionDataRowParser interface {
// Parse returns the converted transaction data row
Parse(ctx core.Context, user *models.User, dataRow CommonDataTableRow, rowId string) (rowData map[TransactionDataTableColumn]string, rowDataValid bool, err error)
}
// commonDataTableToTransactionDataTableWrapper defines the structure of common data table to transaction data table wrapper
type commonDataTableToTransactionDataTableWrapper struct {
innerDataTable CommonDataTable
supportedDataColumns map[TransactionDataTableColumn]bool
rowParser CommonTransactionDataRowParser
}
// commonDataTableToTransactionDataTableWrapperRow defines the data row structure of common data table to transaction data table wrapper
type commonDataTableToTransactionDataTableWrapperRow struct {
transactionDataTable *commonDataTableToTransactionDataTableWrapper
rowData map[TransactionDataTableColumn]string
rowDataValid bool
}
// commonDataTableToTransactionDataTableWrapperRowIterator defines the data row iterator structure of common data table to transaction data table wrapper
type commonDataTableToTransactionDataTableWrapperRowIterator struct {
transactionDataTable *commonDataTableToTransactionDataTableWrapper
innerIterator CommonDataTableRowIterator
}
// HasColumn returns whether the data table has specified column
func (t *commonDataTableToTransactionDataTableWrapper) HasColumn(column TransactionDataTableColumn) bool {
_, exists := t.supportedDataColumns[column]
return exists
}
// TransactionRowCount returns the total count of transaction data row
func (t *commonDataTableToTransactionDataTableWrapper) TransactionRowCount() int {
return t.innerDataTable.DataRowCount()
}
// TransactionRowIterator returns the iterator of transaction data row
func (t *commonDataTableToTransactionDataTableWrapper) TransactionRowIterator() TransactionDataRowIterator {
return &commonDataTableToTransactionDataTableWrapperRowIterator{
transactionDataTable: t,
innerIterator: t.innerDataTable.DataRowIterator(),
}
}
// IsValid returns whether this row is valid data for importing
func (r *commonDataTableToTransactionDataTableWrapperRow) IsValid() bool {
return r.rowDataValid
}
// GetData returns the data in the specified column type
func (r *commonDataTableToTransactionDataTableWrapperRow) GetData(column TransactionDataTableColumn) string {
if !r.rowDataValid {
return ""
}
_, exists := r.transactionDataTable.supportedDataColumns[column]
if !exists {
return ""
}
return r.rowData[column]
}
// HasNext returns whether the iterator does not reach the end
func (t *commonDataTableToTransactionDataTableWrapperRowIterator) HasNext() bool {
return t.innerIterator.HasNext()
}
// Next returns the next transaction data row
func (t *commonDataTableToTransactionDataTableWrapperRowIterator) Next(ctx core.Context, user *models.User) (daraRow TransactionDataRow, err error) {
commonDataRow := t.innerIterator.Next()
if commonDataRow == nil {
return nil, nil
}
rowId := t.innerIterator.CurrentRowId()
rowData, rowDataValid, err := t.transactionDataTable.rowParser.Parse(ctx, user, commonDataRow, rowId)
if err != nil {
log.Errorf(ctx, "[common_data_table_to_transaction_data_table_wrapper.Next] cannot parse data row, because %s", err.Error())
return nil, err
}
return &commonDataTableToTransactionDataTableWrapperRow{
transactionDataTable: t.transactionDataTable,
rowData: rowData,
rowDataValid: rowDataValid,
}, nil
}
// CreateNewTransactionDataTableFromCommonDataTable returns transaction data table from Common data table
func CreateNewTransactionDataTableFromCommonDataTable(dataTable CommonDataTable, supportedDataColumns map[TransactionDataTableColumn]bool, rowParser CommonTransactionDataRowParser) TransactionDataTable {
return &commonDataTableToTransactionDataTableWrapper{
innerDataTable: dataTable,
supportedDataColumns: supportedDataColumns,
rowParser: rowParser,
}
}
@@ -1,114 +0,0 @@
package datatable
import (
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
// CommonTransactionDataTable defines the structure of common transaction data table
type CommonTransactionDataTable struct {
innerDataTable CommonDataTable
supportedDataColumns map[TransactionDataTableColumn]bool
rowParser CommonTransactionDataRowParser
}
// CommonTransactionDataRow defines the structure of common transaction data row
type CommonTransactionDataRow struct {
transactionDataTable *CommonTransactionDataTable
rowData map[TransactionDataTableColumn]string
rowDataValid bool
}
// CommonTransactionDataRowIterator defines the structure of common transaction data row iterator
type CommonTransactionDataRowIterator struct {
transactionDataTable *CommonTransactionDataTable
innerIterator CommonDataRowIterator
}
// CommonTransactionDataRowParser defines the structure of common transaction data row parser
type CommonTransactionDataRowParser interface {
// Parse returns the converted transaction data row
Parse(ctx core.Context, user *models.User, dataTable *CommonTransactionDataTable, dataRow CommonDataRow, rowId string) (rowData map[TransactionDataTableColumn]string, rowDataValid bool, err error)
}
// HasColumn returns whether the data table has specified column
func (t *CommonTransactionDataTable) HasColumn(column TransactionDataTableColumn) bool {
_, exists := t.supportedDataColumns[column]
return exists
}
// HasOriginalColumn returns whether the original data table has specified column name
func (t *CommonTransactionDataTable) HasOriginalColumn(columnName string) bool {
return columnName != "" && t.innerDataTable.HasColumn(columnName)
}
// TransactionRowCount returns the total count of transaction data row
func (t *CommonTransactionDataTable) TransactionRowCount() int {
return t.innerDataTable.DataRowCount()
}
// TransactionRowIterator returns the iterator of transaction data row
func (t *CommonTransactionDataTable) TransactionRowIterator() TransactionDataRowIterator {
return &CommonTransactionDataRowIterator{
transactionDataTable: t,
innerIterator: t.innerDataTable.DataRowIterator(),
}
}
// IsValid returns whether this row is valid data for importing
func (r *CommonTransactionDataRow) IsValid() bool {
return r.rowDataValid
}
// GetData returns the data in the specified column type
func (r *CommonTransactionDataRow) GetData(column TransactionDataTableColumn) string {
if !r.rowDataValid {
return ""
}
_, exists := r.transactionDataTable.supportedDataColumns[column]
if !exists {
return ""
}
return r.rowData[column]
}
// HasNext returns whether the iterator does not reach the end
func (t *CommonTransactionDataRowIterator) HasNext() bool {
return t.innerIterator.HasNext()
}
// Next returns the next transaction data row
func (t *CommonTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow TransactionDataRow, err error) {
commonRow := t.innerIterator.Next()
if commonRow == nil {
return nil, nil
}
rowId := t.innerIterator.CurrentRowId()
rowData, rowDataValid, err := t.transactionDataTable.rowParser.Parse(ctx, user, t.transactionDataTable, commonRow, rowId)
if err != nil {
log.Errorf(ctx, "[common_transaction_data_table.Next] cannot parse data row, because %s", err.Error())
return nil, err
}
return &CommonTransactionDataRow{
transactionDataTable: t.transactionDataTable,
rowData: rowData,
rowDataValid: rowDataValid,
}, nil
}
// CreateNewCommonTransactionDataTable returns transaction data table from Common data table
func CreateNewCommonTransactionDataTable(dataTable CommonDataTable, supportedDataColumns map[TransactionDataTableColumn]bool, rowParser CommonTransactionDataRowParser) *CommonTransactionDataTable {
return &CommonTransactionDataTable{
innerDataTable: dataTable,
supportedDataColumns: supportedDataColumns,
rowParser: rowParser,
}
}
@@ -1,107 +0,0 @@
package datatable
// ImportedCommonDataTable defines the structure of imported common data table
type ImportedCommonDataTable struct {
innerDataTable ImportedDataTable
dataColumnIndexes map[string]int
}
// ImportedCommonDataRow defines the structure of imported common data row
type ImportedCommonDataRow struct {
rowData map[string]string
}
// ImportedCommonDataRowIterator defines the structure of imported common data row iterator
type ImportedCommonDataRowIterator struct {
commonDataTable *ImportedCommonDataTable
innerIterator ImportedDataRowIterator
}
// HeaderColumnCount returns the total count of column in header row
func (t *ImportedCommonDataTable) HeaderColumnCount() int {
return len(t.innerDataTable.HeaderColumnNames())
}
// HasColumn returns whether the data table has specified column name
func (t *ImportedCommonDataTable) HasColumn(columnName string) bool {
index, exists := t.dataColumnIndexes[columnName]
return exists && index >= 0
}
// DataRowCount returns the total count of common data row
func (t *ImportedCommonDataTable) DataRowCount() int {
return t.innerDataTable.DataRowCount()
}
// DataRowIterator returns the iterator of common data row
func (t *ImportedCommonDataTable) DataRowIterator() CommonDataRowIterator {
return &ImportedCommonDataRowIterator{
commonDataTable: t,
innerIterator: t.innerDataTable.DataRowIterator(),
}
}
// HasData returns whether the common data row has specified column data
func (r *ImportedCommonDataRow) HasData(columnName string) bool {
_, exists := r.rowData[columnName]
return exists
}
// ColumnCount returns the total count of column in this data row
func (r *ImportedCommonDataRow) ColumnCount() int {
return len(r.rowData)
}
// GetData returns the data in the specified column name
func (r *ImportedCommonDataRow) GetData(columnName string) string {
return r.rowData[columnName]
}
// HasNext returns whether the iterator does not reach the end
func (t *ImportedCommonDataRowIterator) HasNext() bool {
return t.innerIterator.HasNext()
}
// CurrentRowId returns current row id
func (t *ImportedCommonDataRowIterator) CurrentRowId() string {
return t.innerIterator.CurrentRowId()
}
// Next returns the next common data row
func (t *ImportedCommonDataRowIterator) Next() CommonDataRow {
importedRow := t.innerIterator.Next()
if importedRow == nil {
return nil
}
rowData := make(map[string]string, len(t.commonDataTable.dataColumnIndexes))
for column, columnIndex := range t.commonDataTable.dataColumnIndexes {
if columnIndex < 0 || columnIndex >= importedRow.ColumnCount() {
continue
}
value := importedRow.GetData(columnIndex)
rowData[column] = value
}
return &ImportedCommonDataRow{
rowData: rowData,
}
}
// CreateNewImportedCommonDataTable returns common data table from imported data table
func CreateNewImportedCommonDataTable(dataTable ImportedDataTable) *ImportedCommonDataTable {
headerLineItems := dataTable.HeaderColumnNames()
dataColumnIndexes := make(map[string]int, len(headerLineItems))
for i := 0; i < len(headerLineItems); i++ {
dataColumnIndexes[headerLineItems[i]] = i
}
return &ImportedCommonDataTable{
innerDataTable: dataTable,
dataColumnIndexes: dataColumnIndexes,
}
}
@@ -73,3 +73,6 @@ const (
TRANSACTION_DATA_TABLE_TAGS TransactionDataTableColumn = 13
TRANSACTION_DATA_TABLE_DESCRIPTION TransactionDataTableColumn = 14
)
// TRANSACTION_DATA_TABLE_TIMEZONE_NOT_AVAILABLE represents the constant for timezone not available
const TRANSACTION_DATA_TABLE_TIMEZONE_NOT_AVAILABLE = "TIMEZONE_NOT_AVAILABLE"
@@ -14,6 +14,7 @@ type defaultTransactionDataPlainTextConverter struct {
const ezbookkeepingLineSeparator = "\n"
const ezbookkeepingGeoLocationSeparator = " "
const ezbookkeepingGeoLocationOrder = converter.TRANSACTION_GEO_LOCATION_ORDER_LONGITUDE_LATITUDE
const ezbookkeepingTagSeparator = ";"
var ezbookkeepingDataColumnNameMapping = map[datatable.TransactionDataTableColumn]string{
@@ -94,11 +95,12 @@ func (c *defaultTransactionDataPlainTextConverter) ParseImportedData(ctx core.Co
return nil, nil, nil, nil, nil, nil, err
}
transactionDataTable := datatable.CreateNewImportedTransactionDataTable(dataTable, ezbookkeepingDataColumnNameMapping)
transactionDataTable := datatable.CreateNewTransactionDataTableFromBasicDataTable(dataTable, ezbookkeepingDataColumnNameMapping)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(
ezbookkeepingTransactionTypeNameMapping,
ezbookkeepingGeoLocationSeparator,
ezbookkeepingGeoLocationOrder,
ezbookkeepingTagSeparator,
)
@@ -52,7 +52,7 @@ func (t *defaultPlainTextDataTable) HeaderColumnNames() []string {
}
// DataRowIterator returns the iterator of data row
func (t *defaultPlainTextDataTable) DataRowIterator() datatable.ImportedDataRowIterator {
func (t *defaultPlainTextDataTable) DataRowIterator() datatable.BasicDataTableRowIterator {
return &defaultPlainTextDataRowIterator{
dataTable: t,
currentIndex: 0,
@@ -83,8 +83,8 @@ func (t *defaultPlainTextDataRowIterator) CurrentRowId() string {
return fmt.Sprintf("line#%d", t.currentIndex)
}
// Next returns the next imported data row
func (t *defaultPlainTextDataRowIterator) Next() datatable.ImportedDataRow {
// Next returns the next basic data row
func (t *defaultPlainTextDataRowIterator) Next() datatable.BasicDataTableRow {
if t.currentIndex+1 >= len(t.dataTable.allLines) {
return nil
}
@@ -106,6 +106,7 @@ type customTransactionDataDsvFileImporter struct {
amountDecimalSeparator string
amountDigitGroupingSymbol string
geoLocationSeparator string
geoLocationOrder converter.TransactionGeoLocationOrder
transactionTagSeparator string
}
@@ -156,9 +157,9 @@ func (c *customTransactionDataDsvFileImporter) ParseImportedData(ctx core.Contex
allLines = append([][]string{{}}, allLines...)
}
dataTable := csvconverter.CreateNewCustomCsvImportedDataTable(allLines)
dataTable := csvconverter.CreateNewCustomCsvBasicDataTable(allLines)
transactionDataTable := CreateNewCustomPlainTextDataTable(dataTable, c.columnIndexMapping, c.transactionTypeNameMapping, c.timeFormat, c.timezoneFormat, c.amountDecimalSeparator, c.amountDigitGroupingSymbol)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(customTransactionTypeNameMapping, c.geoLocationSeparator, c.transactionTagSeparator)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(customTransactionTypeNameMapping, c.geoLocationSeparator, c.geoLocationOrder, c.transactionTagSeparator)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -190,7 +191,7 @@ func CreateNewCustomTransactionDataDsvFileParser(fileType string, fileEncoding s
}
// CreateNewCustomTransactionDataDsvFileImporter returns a new custom dsv importer for transaction data
func CreateNewCustomTransactionDataDsvFileImporter(fileType string, fileEncoding string, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, hasHeaderLine bool, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string, geoLocationSeparator string, transactionTagSeparator string) (converter.TransactionDataImporter, error) {
func CreateNewCustomTransactionDataDsvFileImporter(fileType string, fileEncoding string, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, hasHeaderLine bool, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string, geoLocationSeparator string, geoLocationOrder string, transactionTagSeparator string) (converter.TransactionDataImporter, error) {
separator, exists := supportedFileTypeSeparators[fileType]
if !exists {
@@ -203,6 +204,13 @@ func CreateNewCustomTransactionDataDsvFileImporter(fileType string, fileEncoding
return nil, errs.ErrImportFileEncodingNotSupported
}
if geoLocationOrder == "" {
geoLocationOrder = string(converter.TRANSACTION_GEO_LOCATION_ORDER_LONGITUDE_LATITUDE)
} else if geoLocationOrder != string(converter.TRANSACTION_GEO_LOCATION_ORDER_LONGITUDE_LATITUDE) &&
geoLocationOrder != string(converter.TRANSACTION_GEO_LOCATION_ORDER_LATITUDE_LONGITUDE) {
return nil, errs.ErrImportFileTypeNotSupported
}
if _, exists = columnIndexMapping[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME]; !exists {
return nil, errs.ErrMissingRequiredFieldInHeaderRow
}
@@ -226,6 +234,7 @@ func CreateNewCustomTransactionDataDsvFileImporter(fileType string, fileEncoding
amountDecimalSeparator: amountDecimalSeparator,
amountDigitGroupingSymbol: amountDigitGroupingSymbol,
geoLocationSeparator: geoLocationSeparator,
geoLocationOrder: converter.TransactionGeoLocationOrder(geoLocationOrder),
transactionTagSeparator: transactionTagSeparator,
}, nil
}
@@ -77,7 +77,7 @@ func TestCustomTransactionDataDsvFileImporter_MinimumValidData(t *testing.T) {
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", ".", "", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", ".", "", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -168,7 +168,7 @@ func TestCustomTransactionDataDsvFileImporter_WithAllSupportedColumns(t *testing
"Expense": models.TRANSACTION_TYPE_EXPENSE,
"Transfer": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, true, "YYYY-MM-DD HH:mm:ss", "", ".", "", " ", ";")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, true, "YYYY-MM-DD HH:mm:ss", "", ".", "", " ", "lonlat", ";")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -261,7 +261,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidTime(t *testing.T) {
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -292,7 +292,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseTransactionWithoutType(t *tes
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -316,7 +316,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidType(t *testing.T) {
transactionTypeMapping := map[string]models.TransactionType{
"B": 0,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -340,7 +340,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseTimeWithTimezone(t *testing.T
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ssZ", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ssZ", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -378,7 +378,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseTimeWithTimezone2(t *testing.
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ssZZ", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ssZZ", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -417,7 +417,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseValidTimezone(t *testing.T) {
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -456,7 +456,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseValidTimezone2(t *testing.T)
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "ZZ", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "ZZ", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -495,7 +495,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidTimezoneFormat(t *test
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "z", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "z", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -520,7 +520,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidTimezone(t *testing.T)
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -549,7 +549,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidTimezone2(t *testing.T
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "ZZ", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "ZZ", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -577,7 +577,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseAmountWithCustomFormat(t *tes
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ",", ".", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ",", ".", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -603,7 +603,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidAmountWithCustomFormat
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", ",", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", ",", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -627,7 +627,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidAmountWithCustomFormat
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ",", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_tsv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ",", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -655,7 +655,7 @@ func TestCustomTransactionDataDsvFileImporter_ParsePrimaryCategory(t *testing.T)
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -724,7 +724,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseValidAccountCurrency(t *testi
"B": models.TRANSACTION_TYPE_MODIFY_BALANCE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -767,7 +767,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidAccountCurrency(t *tes
"B": models.TRANSACTION_TYPE_MODIFY_BALANCE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -803,7 +803,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseNotSupportedCurrency(t *testi
"B": models.TRANSACTION_TYPE_MODIFY_BALANCE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -835,7 +835,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseValidAmount(t *testing.T) {
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -886,7 +886,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidAmount(t *testing.T) {
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -917,7 +917,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseNoAmount2(t *testing.T) {
"E": models.TRANSACTION_TYPE_EXPENSE,
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -952,7 +952,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseValidGeographicLocation(t *te
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", ";", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", ";", "lonlat", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -981,7 +981,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseInvalidGeographicLocation(t *
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", " ", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", " ", "lonlat", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -1013,7 +1013,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseTag(t *testing.T) {
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", ";")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", ";")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -1053,7 +1053,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseTagWithoutSeparator(t *testin
transactionTypeMapping := map[string]models.TransactionType{
"E": models.TRANSACTION_TYPE_EXPENSE,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -1084,7 +1084,7 @@ func TestCustomTransactionDataDsvFileImporter_ParseDescription(t *testing.T) {
transactionTypeMapping := map[string]models.TransactionType{
"T": models.TRANSACTION_TYPE_TRANSFER,
}
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
converter, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.Nil(t, err)
context := core.NewNullContext()
@@ -1111,7 +1111,7 @@ func TestCustomTransactionDataDsvFileImporter_InvalidSeparator(t *testing.T) {
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: 1,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: 2,
}
_, err := CreateNewCustomTransactionDataDsvFileImporter("test", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
_, err := CreateNewCustomTransactionDataDsvFileImporter("test", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.EqualError(t, err, errs.ErrImportFileTypeNotSupported.Message)
}
@@ -1124,7 +1124,7 @@ func TestCustomTransactionDataDsvFileImporter_InvalidFileEncoding(t *testing.T)
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: 1,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: 2,
}
_, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "ascii", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
_, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "ascii", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.EqualError(t, err, errs.ErrImportFileEncodingNotSupported.Message)
}
@@ -1138,7 +1138,7 @@ func TestCustomTransactionDataDsvFileImporter_MissingRequiredColumn(t *testing.T
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: 0,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: 1,
}
_, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
_, err := CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Type Column
@@ -1146,7 +1146,7 @@ func TestCustomTransactionDataDsvFileImporter_MissingRequiredColumn(t *testing.T
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: 0,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: 1,
}
_, err = CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
_, err = CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Amount Column
@@ -1154,6 +1154,6 @@ func TestCustomTransactionDataDsvFileImporter_MissingRequiredColumn(t *testing.T
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: 0,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: 1,
}
_, err = CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "")
_, err = CreateNewCustomTransactionDataDsvFileImporter("custom_csv", "utf-8", columnIndexMapping, transactionTypeMapping, false, "YYYY-MM-DD HH:mm:ss", "", ".", "", "", "", "")
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
}
@@ -14,7 +14,7 @@ import (
// customPlainTextDataTable defines the structure of custom plain text transaction data table
type customPlainTextDataTable struct {
innerDataTable datatable.ImportedDataTable
innerDataTable datatable.BasicDataTable
columnIndexMapping map[datatable.TransactionDataTableColumn]int
transactionTypeNameMapping map[string]models.TransactionType
timeFormat string
@@ -34,7 +34,7 @@ type customPlainTextDataRow struct {
// customPlainTextDataRowIterator defines the structure of custom plain text transaction data row iterator
type customPlainTextDataRowIterator struct {
transactionDataTable *customPlainTextDataTable
innerIterator datatable.ImportedDataRowIterator
innerIterator datatable.BasicDataTableRowIterator
}
// HasColumn returns whether the data table has specified column
@@ -105,7 +105,7 @@ func (t *customPlainTextDataRowIterator) Next(ctx core.Context, user *models.Use
}, nil
}
func (t *customPlainTextDataRowIterator) parseTransaction(ctx core.Context, user *models.User, row datatable.ImportedDataRow) (map[datatable.TransactionDataTableColumn]string, bool, error) {
func (t *customPlainTextDataRowIterator) parseTransaction(ctx core.Context, user *models.User, row datatable.BasicDataTableRow) (map[datatable.TransactionDataTableColumn]string, bool, error) {
rowData := make(map[datatable.TransactionDataTableColumn]string, len(t.transactionDataTable.columnIndexMapping))
for column, columnIndex := range t.transactionDataTable.columnIndexMapping {
@@ -236,7 +236,7 @@ func (t *customPlainTextDataRowIterator) parseAmount(ctx core.Context, amountVal
}
// CreateNewCustomPlainTextDataTable returns transaction data table from imported data table
func CreateNewCustomPlainTextDataTable(dataTable datatable.ImportedDataTable, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string) *customPlainTextDataTable {
func CreateNewCustomPlainTextDataTable(dataTable datatable.BasicDataTable, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string) *customPlainTextDataTable {
timeFormatIncludeTimezone := strings.Contains(timeFormat, "z") || strings.Contains(timeFormat, "Z")
return &customPlainTextDataTable{
@@ -10,27 +10,27 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
// ExcelMSCFBFileImportedDataTable defines the structure of excel (microsoft compound file binary) file data table
type ExcelMSCFBFileImportedDataTable struct {
// ExcelMSCFBFileBasicDataTable defines the structure of excel (microsoft compound file binary) file data table
type ExcelMSCFBFileBasicDataTable struct {
workbook *xls.WorkBook
headerLineColumnNames []string
}
// ExcelMSCFBFileDataRow defines the structure of excel (microsoft compound file binary) file data table row
type ExcelMSCFBFileDataRow struct {
// ExcelMSCFBFileBasicDataTableRow defines the structure of excel (microsoft compound file binary) file data table row
type ExcelMSCFBFileBasicDataTableRow struct {
sheet *xls.WorkSheet
rowIndex int
}
// ExcelMSCFBFileDataRowIterator defines the structure of excel (microsoft compound file binary) file data table row iterator
type ExcelMSCFBFileDataRowIterator struct {
dataTable *ExcelMSCFBFileImportedDataTable
// ExcelMSCFBFileBasicDataTableRowIterator defines the structure of excel (microsoft compound file binary) file data table row iterator
type ExcelMSCFBFileBasicDataTableRowIterator struct {
dataTable *ExcelMSCFBFileBasicDataTable
currentSheetIndex int
currentRowIndexInSheet uint16
}
// DataRowCount returns the total count of data row
func (t *ExcelMSCFBFileImportedDataTable) DataRowCount() int {
func (t *ExcelMSCFBFileBasicDataTable) DataRowCount() int {
totalDataRowCount := 0
for i := 0; i < t.workbook.NumSheets(); i++ {
@@ -47,13 +47,13 @@ func (t *ExcelMSCFBFileImportedDataTable) DataRowCount() int {
}
// HeaderColumnNames returns the header column name list
func (t *ExcelMSCFBFileImportedDataTable) HeaderColumnNames() []string {
func (t *ExcelMSCFBFileBasicDataTable) HeaderColumnNames() []string {
return t.headerLineColumnNames
}
// DataRowIterator returns the iterator of data row
func (t *ExcelMSCFBFileImportedDataTable) DataRowIterator() datatable.ImportedDataRowIterator {
return &ExcelMSCFBFileDataRowIterator{
func (t *ExcelMSCFBFileBasicDataTable) DataRowIterator() datatable.BasicDataTableRowIterator {
return &ExcelMSCFBFileBasicDataTableRowIterator{
dataTable: t,
currentSheetIndex: 0,
currentRowIndexInSheet: 0,
@@ -61,19 +61,19 @@ func (t *ExcelMSCFBFileImportedDataTable) DataRowIterator() datatable.ImportedDa
}
// ColumnCount returns the total count of column in this data row
func (r *ExcelMSCFBFileDataRow) ColumnCount() int {
func (r *ExcelMSCFBFileBasicDataTableRow) ColumnCount() int {
row := r.sheet.Row(r.rowIndex)
return row.LastCol() + 1
}
// GetData returns the data in the specified column index
func (r *ExcelMSCFBFileDataRow) GetData(columnIndex int) string {
func (r *ExcelMSCFBFileBasicDataTableRow) GetData(columnIndex int) string {
row := r.sheet.Row(r.rowIndex)
return row.Col(columnIndex)
}
// HasNext returns whether the iterator does not reach the end
func (t *ExcelMSCFBFileDataRowIterator) HasNext() bool {
func (t *ExcelMSCFBFileBasicDataTableRowIterator) HasNext() bool {
workbook := t.dataTable.workbook
if t.currentSheetIndex >= workbook.NumSheets() {
@@ -100,12 +100,12 @@ func (t *ExcelMSCFBFileDataRowIterator) HasNext() bool {
}
// CurrentRowId returns current index
func (t *ExcelMSCFBFileDataRowIterator) CurrentRowId() string {
return fmt.Sprintf("table#%d-row#%d", t.currentSheetIndex, t.currentRowIndexInSheet)
func (t *ExcelMSCFBFileBasicDataTableRowIterator) CurrentRowId() string {
return fmt.Sprintf("sheet#%d-row#%d", t.currentSheetIndex, t.currentRowIndexInSheet)
}
// Next returns the next imported data row
func (t *ExcelMSCFBFileDataRowIterator) Next() datatable.ImportedDataRow {
// Next returns the next basic data row
func (t *ExcelMSCFBFileBasicDataTableRowIterator) Next() datatable.BasicDataTableRow {
workbook := t.dataTable.workbook
currentRowIndexInTable := t.currentRowIndexInSheet
@@ -133,14 +133,14 @@ func (t *ExcelMSCFBFileDataRowIterator) Next() datatable.ImportedDataRow {
return nil
}
return &ExcelMSCFBFileDataRow{
return &ExcelMSCFBFileBasicDataTableRow{
sheet: currentSheet,
rowIndex: int(t.currentRowIndexInSheet),
}
}
// CreateNewExcelMSCFBFileImportedDataTable returns excel (microsoft compound file binary) data table by file binary data
func CreateNewExcelMSCFBFileImportedDataTable(data []byte) (*ExcelMSCFBFileImportedDataTable, error) {
// CreateNewExcelMSCFBFileBasicDataTable returns excel (microsoft compound file binary) data table by file binary data
func CreateNewExcelMSCFBFileBasicDataTable(data []byte) (datatable.BasicDataTable, error) {
reader := bytes.NewReader(data)
workbook, err := xls.OpenReader(reader, "")
@@ -184,7 +184,7 @@ func CreateNewExcelMSCFBFileImportedDataTable(data []byte) (*ExcelMSCFBFileImpor
}
}
return &ExcelMSCFBFileImportedDataTable{
return &ExcelMSCFBFileBasicDataTable{
workbook: workbook,
headerLineColumnNames: headerRowItems,
}, nil
@@ -9,63 +9,63 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
func TestExcelMSCFBFileImportedDataTableDataRowCount(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableDataRowCount(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 2, datatable.DataRowCount())
}
func TestExcelMSCFBFileImportedDataTableDataRowCount_MultipleSheets(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableDataRowCount_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 5, datatable.DataRowCount())
}
func TestExcelMSCFBFileImportedDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/only_one_row_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestExcelMSCFBFileImportedDataTableDataRowCount_EmptyContent(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableDataRowCount_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestExcelMSCFBFileImportedDataTableHeaderColumnNames(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableHeaderColumnNames(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.EqualValues(t, []string{"A1", "B1", "C1"}, datatable.HeaderColumnNames())
}
func TestExcelMSCFBFileImportedDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.Nil(t, datatable.HeaderColumnNames())
}
func TestExcelMSCFBFileDataRowIterator(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowIterator(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.True(t, iterator.HasNext())
@@ -86,11 +86,11 @@ func TestExcelMSCFBFileDataRowIterator(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelMSCFBFileDataRowIterator_MultipleSheets(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowIterator_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.True(t, iterator.HasNext())
@@ -123,11 +123,11 @@ func TestExcelMSCFBFileDataRowIterator_MultipleSheets(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelMSCFBFileDataRowIterator_OnlyHeaderLine(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowIterator_OnlyHeaderLine(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/only_one_row_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.False(t, iterator.HasNext())
@@ -140,11 +140,11 @@ func TestExcelMSCFBFileDataRowIterator_OnlyHeaderLine(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelMSCFBFileDataRowIterator_EmptyContent(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowIterator_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.False(t, iterator.HasNext())
@@ -157,11 +157,11 @@ func TestExcelMSCFBFileDataRowIterator_EmptyContent(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelMSCFBFileDataRowColumnCount(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowColumnCount(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
@@ -171,11 +171,11 @@ func TestExcelMSCFBFileDataRowColumnCount(t *testing.T) {
assert.EqualValues(t, 4, row2.ColumnCount())
}
func TestExcelMSCFBFileDataRowGetData(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowGetData(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
@@ -189,22 +189,22 @@ func TestExcelMSCFBFileDataRowGetData(t *testing.T) {
assert.Equal(t, "C3", row2.GetData(2))
}
func TestExcelMSCFBFileDataRowGetData_GetNotExistedColumnData(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowGetData_GetNotExistedColumnData(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
assert.Equal(t, "", row1.GetData(3))
}
func TestExcelMSCFBFileDataRowGetData_MultipleSheets(t *testing.T) {
func TestExcelMSCFBFileBasicDataTableRowGetData_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xls")
assert.Nil(t, err)
datatable, err := CreateNewExcelMSCFBFileImportedDataTable(testdata)
datatable, err := CreateNewExcelMSCFBFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
sheet1Row1 := iterator.Next()
@@ -237,10 +237,10 @@ func TestExcelMSCFBFileDataRowGetData_MultipleSheets(t *testing.T) {
assert.Equal(t, "5-C3", sheet5Row2.GetData(2))
}
func TestCreateNewExcelMSCFBFileImportedDataTable_MultipleSheetsWithDifferentHeaders(t *testing.T) {
func TestCreateNewExcelMSCFBFileBasicDataTable_MultipleSheetsWithDifferentHeaders(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_with_different_header_row_excel_file.xls")
assert.Nil(t, err)
_, err = CreateNewExcelMSCFBFileImportedDataTable(testdata)
_, err = CreateNewExcelMSCFBFileBasicDataTable(testdata)
assert.EqualError(t, err, errs.ErrFieldsInMultiTableAreDifferent.Message)
}
@@ -16,28 +16,28 @@ type excelOOXMLSheet struct {
allData [][]string
}
// ExcelOOXMLFileImportedDataTable defines the structure of excel (Office Open XML) file data table
type ExcelOOXMLFileImportedDataTable struct {
// ExcelOOXMLFileBasicDataTable defines the structure of excel (Office Open XML) file data table
type ExcelOOXMLFileBasicDataTable struct {
sheets []*excelOOXMLSheet
headerLineColumnNames []string
}
// ExcelOOXMLFileDataRow defines the structure of excel (Office Open XML) file data table row
type ExcelOOXMLFileDataRow struct {
// ExcelOOXMLFileBasicDataTableRow defines the structure of excel (Office Open XML) file data table row
type ExcelOOXMLFileBasicDataTableRow struct {
sheet *excelOOXMLSheet
rowData []string
rowIndex int
}
// ExcelOOXMLFileDataRowIterator defines the structure of excel (Office Open XML) file data table row iterator
type ExcelOOXMLFileDataRowIterator struct {
dataTable *ExcelOOXMLFileImportedDataTable
// ExcelOOXMLFileBasicDataTableRowIterator defines the structure of excel (Office Open XML) file data table row iterator
type ExcelOOXMLFileBasicDataTableRowIterator struct {
dataTable *ExcelOOXMLFileBasicDataTable
currentSheetIndex int
currentRowIndexInSheet int
}
// DataRowCount returns the total count of data row
func (t *ExcelOOXMLFileImportedDataTable) DataRowCount() int {
func (t *ExcelOOXMLFileBasicDataTable) DataRowCount() int {
totalDataRowCount := 0
for i := 0; i < len(t.sheets); i++ {
@@ -54,13 +54,13 @@ func (t *ExcelOOXMLFileImportedDataTable) DataRowCount() int {
}
// HeaderColumnNames returns the header column name list
func (t *ExcelOOXMLFileImportedDataTable) HeaderColumnNames() []string {
func (t *ExcelOOXMLFileBasicDataTable) HeaderColumnNames() []string {
return t.headerLineColumnNames
}
// DataRowIterator returns the iterator of data row
func (t *ExcelOOXMLFileImportedDataTable) DataRowIterator() datatable.ImportedDataRowIterator {
return &ExcelOOXMLFileDataRowIterator{
func (t *ExcelOOXMLFileBasicDataTable) DataRowIterator() datatable.BasicDataTableRowIterator {
return &ExcelOOXMLFileBasicDataTableRowIterator{
dataTable: t,
currentSheetIndex: 0,
currentRowIndexInSheet: 0,
@@ -68,12 +68,12 @@ func (t *ExcelOOXMLFileImportedDataTable) DataRowIterator() datatable.ImportedDa
}
// ColumnCount returns the total count of column in this data row
func (r *ExcelOOXMLFileDataRow) ColumnCount() int {
func (r *ExcelOOXMLFileBasicDataTableRow) ColumnCount() int {
return len(r.rowData)
}
// GetData returns the data in the specified column index
func (r *ExcelOOXMLFileDataRow) GetData(columnIndex int) string {
func (r *ExcelOOXMLFileBasicDataTableRow) GetData(columnIndex int) string {
if columnIndex < 0 || columnIndex >= len(r.rowData) {
return ""
}
@@ -82,7 +82,7 @@ func (r *ExcelOOXMLFileDataRow) GetData(columnIndex int) string {
}
// HasNext returns whether the iterator does not reach the end
func (t *ExcelOOXMLFileDataRowIterator) HasNext() bool {
func (t *ExcelOOXMLFileBasicDataTableRowIterator) HasNext() bool {
sheets := t.dataTable.sheets
if t.currentSheetIndex >= len(sheets) {
@@ -109,12 +109,12 @@ func (t *ExcelOOXMLFileDataRowIterator) HasNext() bool {
}
// CurrentRowId returns current index
func (t *ExcelOOXMLFileDataRowIterator) CurrentRowId() string {
return fmt.Sprintf("table#%d-row#%d", t.currentSheetIndex, t.currentRowIndexInSheet)
func (t *ExcelOOXMLFileBasicDataTableRowIterator) CurrentRowId() string {
return fmt.Sprintf("sheet#%d-row#%d", t.currentSheetIndex, t.currentRowIndexInSheet)
}
// Next returns the next imported data row
func (t *ExcelOOXMLFileDataRowIterator) Next() datatable.ImportedDataRow {
// Next returns the next basic data row
func (t *ExcelOOXMLFileBasicDataTableRowIterator) Next() datatable.BasicDataTableRow {
sheets := t.dataTable.sheets
currentRowIndexInTable := t.currentRowIndexInSheet
@@ -142,15 +142,15 @@ func (t *ExcelOOXMLFileDataRowIterator) Next() datatable.ImportedDataRow {
return nil
}
return &ExcelOOXMLFileDataRow{
return &ExcelOOXMLFileBasicDataTableRow{
sheet: currentSheet,
rowData: currentSheet.allData[t.currentRowIndexInSheet],
rowIndex: t.currentRowIndexInSheet,
}
}
// CreateNewExcelOOXMLFileImportedDataTable returns excel (Office Open XML) data table by file binary data
func CreateNewExcelOOXMLFileImportedDataTable(data []byte) (*ExcelOOXMLFileImportedDataTable, error) {
// CreateNewExcelOOXMLFileBasicDataTable returns excel (Office Open XML) data table by file binary data
func CreateNewExcelOOXMLFileBasicDataTable(data []byte) (datatable.BasicDataTable, error) {
reader := bytes.NewReader(data)
file, err := excelize.OpenReader(reader)
@@ -204,7 +204,7 @@ func CreateNewExcelOOXMLFileImportedDataTable(data []byte) (*ExcelOOXMLFileImpor
})
}
return &ExcelOOXMLFileImportedDataTable{
return &ExcelOOXMLFileBasicDataTable{
sheets: sheets,
headerLineColumnNames: headerRowItems,
}, nil
@@ -9,63 +9,63 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
func TestExcelOOXMLFileImportedDataTableDataRowCount(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableDataRowCount(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 2, datatable.DataRowCount())
}
func TestExcelOOXMLFileImportedDataTableDataRowCount_MultipleSheets(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableDataRowCount_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 5, datatable.DataRowCount())
}
func TestExcelOOXMLFileImportedDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableDataRowCount_OnlyHeaderLine(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/only_one_row_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestExcelOOXMLFileImportedDataTableDataRowCount_EmptyContent(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableDataRowCount_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.Nil(t, err)
assert.Equal(t, 0, datatable.DataRowCount())
}
func TestExcelOOXMLFileImportedDataTableHeaderColumnNames(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableHeaderColumnNames(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.EqualValues(t, []string{"A1", "B1", "C1"}, datatable.HeaderColumnNames())
}
func TestExcelOOXMLFileImportedDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
func TestExcelOOXMLFileBasicDataTableHeaderColumnNames_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.Nil(t, datatable.HeaderColumnNames())
}
func TestExcelOOXMLFileDataRowIterator(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowIterator(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.True(t, iterator.HasNext())
@@ -86,11 +86,11 @@ func TestExcelOOXMLFileDataRowIterator(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelOOXMLFileDataRowIterator_MultipleSheets(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowIterator_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.True(t, iterator.HasNext())
@@ -123,11 +123,11 @@ func TestExcelOOXMLFileDataRowIterator_MultipleSheets(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelOOXMLFileDataRowIterator_OnlyHeaderLine(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowIterator_OnlyHeaderLine(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/only_one_row_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.False(t, iterator.HasNext())
@@ -140,11 +140,11 @@ func TestExcelOOXMLFileDataRowIterator_OnlyHeaderLine(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelOOXMLFileDataRowIterator_EmptyContent(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowIterator_EmptyContent(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/empty_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
assert.False(t, iterator.HasNext())
@@ -157,11 +157,11 @@ func TestExcelOOXMLFileDataRowIterator_EmptyContent(t *testing.T) {
assert.False(t, iterator.HasNext())
}
func TestExcelOOXMLFileDataRowColumnCount(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowColumnCount(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
@@ -171,11 +171,11 @@ func TestExcelOOXMLFileDataRowColumnCount(t *testing.T) {
assert.EqualValues(t, 3, row2.ColumnCount())
}
func TestExcelOOXMLFileDataRowGetData(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowGetData(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
@@ -189,22 +189,22 @@ func TestExcelOOXMLFileDataRowGetData(t *testing.T) {
assert.Equal(t, "C3", row2.GetData(2))
}
func TestExcelOOXMLFileDataRowGetData_GetNotExistedColumnData(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowGetData_GetNotExistedColumnData(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/simple_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
row1 := iterator.Next()
assert.Equal(t, "", row1.GetData(3))
}
func TestExcelOOXMLFileDataRowGetData_MultipleSheets(t *testing.T) {
func TestExcelOOXMLFileBasicDataRowGetData_MultipleSheets(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_excel_file.xlsx")
assert.Nil(t, err)
datatable, err := CreateNewExcelOOXMLFileImportedDataTable(testdata)
datatable, err := CreateNewExcelOOXMLFileBasicDataTable(testdata)
iterator := datatable.DataRowIterator()
sheet1Row1 := iterator.Next()
@@ -237,10 +237,10 @@ func TestExcelOOXMLFileDataRowGetData_MultipleSheets(t *testing.T) {
assert.Equal(t, "5-C3", sheet5Row2.GetData(2))
}
func TestCreateNewExcelOOXMLFileImportedDataTable_MultipleSheetsWithDifferentHeaders(t *testing.T) {
func TestCreateNewExcelOOXMLFileBasicDataTable_MultipleSheetsWithDifferentHeaders(t *testing.T) {
testdata, err := os.ReadFile("../../../testdata/multiple_sheets_with_different_header_row_excel_file.xlsx")
assert.Nil(t, err)
_, err = CreateNewExcelOOXMLFileImportedDataTable(testdata)
_, err = CreateNewExcelOOXMLFileBasicDataTable(testdata)
assert.EqualError(t, err, errs.ErrFieldsInMultiTableAreDifferent.Message)
}
@@ -60,13 +60,13 @@ func (c *feideeMymoneyAppTransactionDataCsvFileImporter) ParseImportedData(ctx c
fallback := unicode.UTF8.NewDecoder()
reader := transform.NewReader(bytes.NewReader(data), unicode.BOMOverride(fallback))
dataTable, err := c.createNewFeideeMymoneyAppImportedDataTable(ctx, reader)
dataTable, err := c.createNewFeideeMymoneyAppBasicDataTable(ctx, reader)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
commonDataTable := datatable.CreateNewImportedCommonDataTable(dataTable)
commonDataTable := datatable.CreateNewCommonDataTableFromBasicDataTable(dataTable)
if !commonDataTable.HasColumn(feideeMymoneyAppTransactionTimeColumnName) ||
!commonDataTable.HasColumn(feideeMymoneyAppTransactionTypeColumnName) ||
@@ -89,7 +89,7 @@ func (c *feideeMymoneyAppTransactionDataCsvFileImporter) ParseImportedData(ctx c
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
func (c *feideeMymoneyAppTransactionDataCsvFileImporter) createNewFeideeMymoneyAppImportedDataTable(ctx core.Context, reader io.Reader) (datatable.ImportedDataTable, error) {
func (c *feideeMymoneyAppTransactionDataCsvFileImporter) createNewFeideeMymoneyAppBasicDataTable(ctx core.Context, reader io.Reader) (datatable.BasicDataTable, error) {
csvReader := csv.NewReader(reader)
csvReader.FieldsPerRecord = -1
@@ -132,7 +132,7 @@ func (c *feideeMymoneyAppTransactionDataCsvFileImporter) createNewFeideeMymoneyA
return nil, errs.ErrNotFoundTransactionDataInFile
}
dataTable := csvdatatable.CreateNewCustomCsvImportedDataTable(allOriginalLines)
dataTable := csvdatatable.CreateNewCustomCsvBasicDataTable(allOriginalLines)
return dataTable, nil
}
@@ -32,14 +32,14 @@ var (
// ParseImportedData returns the imported data by parsing the feidee mymoney (elecloud) transaction xlsx data
func (c *feideeMymoneyElecloudTransactionDataXlsxFileImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
dataTable, err := excel.CreateNewExcelOOXMLFileImportedDataTable(data)
dataTable, err := excel.CreateNewExcelOOXMLFileBasicDataTable(data)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
transactionRowParser := createFeideeMymoneyElecloudTransactionDataRowParser()
transactionDataTable := datatable.CreateNewImportedTransactionDataTableWithRowParser(dataTable, feideeMymoneyElecloudDataColumnNameMapping, transactionRowParser)
transactionDataTable := datatable.CreateNewTransactionDataTableFromBasicDataTableWithRowParser(dataTable, feideeMymoneyElecloudDataColumnNameMapping, transactionRowParser)
dataTableImporter := converter.CreateNewSimpleImporter(feideeMymoneyElecloudTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
@@ -31,14 +31,14 @@ var (
// ParseImportedData returns the imported data by parsing the feidee mymoney (web) transaction xls data
func (c *feideeMymoneyWebTransactionDataXlsFileImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
dataTable, err := excel.CreateNewExcelMSCFBFileImportedDataTable(data)
dataTable, err := excel.CreateNewExcelMSCFBFileBasicDataTable(data)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
transactionRowParser := createFeideeMymoneyTransactionDataRowParser()
transactionDataTable := datatable.CreateNewImportedTransactionDataTableWithRowParser(dataTable, feideeMymoneyWebDataColumnNameMapping, transactionRowParser)
transactionDataTable := datatable.CreateNewTransactionDataTableFromBasicDataTableWithRowParser(dataTable, feideeMymoneyWebDataColumnNameMapping, transactionRowParser)
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(feideeMymoneyTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
@@ -7,21 +7,24 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/converters/csv"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
)
var fireflyIIITransactionDataColumnNameMapping = map[datatable.TransactionDataTableColumn]string{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: "date",
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: "type",
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY: "category",
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME: "source_name",
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY: "currency_code",
datatable.TRANSACTION_DATA_TABLE_AMOUNT: "amount",
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME: "destination_name",
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY: "foreign_currency_code",
datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT: "foreign_amount",
datatable.TRANSACTION_DATA_TABLE_TAGS: "tags",
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION: "description",
var fireflyIIITransactionSupportedColumns = map[datatable.TransactionDataTableColumn]bool{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: true,
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY: true,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT: true,
datatable.TRANSACTION_DATA_TABLE_TAGS: true,
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION: true,
}
var fireflyIIITransactionTypeNameMapping = map[models.TransactionType]string{
@@ -42,15 +45,28 @@ var (
// ParseImportedData returns the imported data by parsing the firefly III transaction csv data
func (c *fireflyIIITransactionDataCsvFileImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
reader := bytes.NewReader(data)
dataTable, err := csv.CreateNewCsvImportedDataTable(ctx, reader)
dataTable, err := csv.CreateNewCsvBasicDataTable(ctx, reader)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
commonDataTable := datatable.CreateNewCommonDataTableFromBasicDataTable(dataTable)
if !commonDataTable.HasColumn(fireflyIIITransactionTimeColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionTypeColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionSourceAccountNameColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionSourceAccountTypeColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionDestinationAccountNameColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionDestinationAccountTypeColumnName) ||
!commonDataTable.HasColumn(fireflyIIITransactionAmountColumnName) {
log.Errorf(ctx, "[fireflyiii_transaction_data_csv_file_importer.ParseImportedData] cannot parse Firefly III csv data, because missing essential columns in header row")
return nil, nil, nil, nil, nil, nil, errs.ErrMissingRequiredFieldInHeaderRow
}
transactionRowParser := createFireflyIIITransactionDataRowParser()
transactionDataTable := datatable.CreateNewImportedTransactionDataTableWithRowParser(dataTable, fireflyIIITransactionDataColumnNameMapping, transactionRowParser)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(fireflyIIITransactionTypeNameMapping, "", ",")
transactionDataTable := datatable.CreateNewTransactionDataTableFromCommonDataTable(commonDataTable, fireflyIIITransactionSupportedColumns, transactionRowParser)
dataTableImporter := converter.CreateNewImporterWithTypeNameMapping(fireflyIIITransactionTypeNameMapping, "", "", ",")
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -20,11 +20,11 @@ func TestFireFlyIIICsvFileConverterParseImportedData_MinimumValidData(t *testing
DefaultCurrency: "CNY",
}
allNewTransactions, allNewAccounts, allNewSubExpenseCategories, allNewSubIncomeCategories, allNewSubTransferCategories, allNewTags, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"+
"Deposit,-0.12,2024-09-01T01:23:45+08:00,\"A revenue account\",\"Test Account\",\"Test Category\"\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"A expense account\",\"Test Category2\"\n"+
"Transfer,-0.05,2024-09-01T23:59:59+08:00,\"Test Account\",\"Test Account2\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
allNewTransactions, allNewAccounts, allNewSubExpenseCategories, allNewSubIncomeCategories, allNewSubTransferCategories, allNewTags, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"+
"Deposit,0.12,2024-09-01T01:23:45+08:00,\"A revenue account\",\"Revenue account\",\"Test Account\",\"Asset account\",\"Test Category\"\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category2\"\n"+
"Transfer,0.05,2024-09-01T23:59:59+08:00,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
@@ -91,16 +91,16 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidTime(t *testing
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-1.00,2024-09-01 12:34:56+08:00,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01 12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTimeInvalid.Message)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidType(t *testing.T) {
func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidTransactionType(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
@@ -109,11 +109,134 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidType(t *testing
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Type,-123.45,2024-09-01T12:34:56+08:00,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
// income transactions
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Deposit,10.00,2024-09-01T12:34:56+08:00,\"A revenue account\",\"Revenue account\",\"Test Account\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Deposit,10.00,2024-09-01T12:34:56+08:00,\"A revenue account\",\"Revenue account\",\"Test Account\",\"Debt\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
// expense transactions
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Debt\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
// opening balance transactions
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",10.00,2024-09-01T12:34:56+08:00,\"Initial balance\",\"Initial balance account\",\"Test Account\",\"Asset account\",\"\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_MODIFY_BALANCE, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
// transfer transactions
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[0].OriginalDestinationAccountName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"Test Account2\",\"Debt\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[0].OriginalDestinationAccountName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Deposit,10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Debt\",\"Test Account2\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[0].OriginalDestinationAccountName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Debt\",\"Test Account2\",\"Debt\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, "Test Account", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "Test Account2", allNewTransactions[0].OriginalDestinationAccountName)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidTransactionType(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Revenue account\",\"Test Account2\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrTransactionTypeInvalid.Message)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseAccountNameAsCategoryName(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "A expense account", allNewTransactions[0].OriginalCategoryName)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Deposit,10.00,2024-09-01T12:34:56+08:00,\"A revenue account\",\"Revenue account\",\"Test Account\",\"Asset account\",\"\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "A revenue account", allNewTransactions[0].OriginalCategoryName)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidTimezone(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
@@ -123,20 +246,20 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidTimezone(t *testi
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56-10:00,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56-10:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, int64(1725230096), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+00:00,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+00:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, int64(1725194096), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+12:45,\"Test Account\",\"A expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-1.00,2024-09-01T12:34:56+12:45,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, int64(1725148196), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
@@ -151,9 +274,9 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidAccountCurrency(t
DefaultCurrency: "CNY",
}
allNewTransactions, allNewAccounts, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"+
"Transfer,-1.23,-1.10,2024-09-01T23:59:59+08:00,USD,EUR,\"Test Account\",\"Test Account2\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
allNewTransactions, allNewAccounts, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"+
"Transfer,1.23,-1.10,2024-09-01T23:59:59+08:00,USD,EUR,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
@@ -169,6 +292,45 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidAccountCurrency(t
assert.Equal(t, "EUR", allNewAccounts[1].Currency)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseValidForeignAmountAndCurrency(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,15.00,2024-09-01T12:34:56+08:00,USD,EUR,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_TRANSFER_OUT, allNewTransactions[0].Type)
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, int64(1500), allNewTransactions[0].RelatedAccountAmount)
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, "EUR", allNewTransactions[0].OriginalDestinationAccountCurrency)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,2024-09-01T12:34:56+08:00,USD,EUR,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, int64(1000), allNewTransactions[0].Amount)
assert.Equal(t, int64(1000), allNewTransactions[0].RelatedAccountAmount)
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, "EUR", allNewTransactions[0].OriginalDestinationAccountCurrency)
allNewTransactions, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,10.00,2024-09-01T12:34:56+08:00,USD,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category\""), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "USD", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, "USD", allNewTransactions[0].OriginalDestinationAccountCurrency)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidAccountCurrency(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
@@ -178,14 +340,14 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidAccountCurrency
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"+
"Transfer,-1.23,-1.10,2024-09-01T23:59:59+08:00,CNY,EUR,\"Test Account\",\"Test Account2\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"+
"Transfer,1.23,1.10,2024-09-01T23:59:59+08:00,CNY,EUR,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"+
"Transfer,-1.23,-1.10,2024-09-01T23:59:59+08:00,CNY,EUR,\"Test Account2\",\"Test Account\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,,2024-09-01T00:00:00+08:00,USD,,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\n"+
"Transfer,1.23,1.10,2024-09-01T23:59:59+08:00,CNY,EUR,\"Test Account2\",\"Asset account\",\"Test Account\",\"Asset account\",\"Test Category3\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
}
@@ -198,12 +360,12 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseNotSupportedCurrency(t
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,,2024-09-01T00:00:00+08:00,XXX,,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,,2024-09-01T00:00:00+08:00,XXX,,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,destination_name,category\n"+
"Transfer,-123.45,-123.45,2024-09-01T23:59:59+08:00,USD,XXX,\"Test Account\",\"Test Account2\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,currency_code,foreign_currency_code,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,123.45,123.45,2024-09-01T23:59:59+08:00,USD,XXX,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
}
@@ -216,12 +378,12 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseInvalidAmount(t *testi
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,category\n"+
"Withdrawal,-123 45,2024-09-01T12:34:56+08:00,\"Test Account\",\"A expense account\",\"Test Category\"\n"), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-123 45,2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\"\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,source_name,destination_name,category\n"+
"Transfer,-123.45,-123 45,2024-09-01T23:59:59+08:00,\"Test Account\",\"Test Account2\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,foreign_amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"Transfer,123.45,123 45,2024-09-01T23:59:59+08:00,\"Test Account\",\"Asset account\",\"Test Account2\",\"Asset account\",\"Test Category2\""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
}
@@ -234,14 +396,37 @@ func TestFireFlyIIICsvFileConverterParseImportedData_ParseDescription(t *testing
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,description,date,source_name,destination_name,category\n"+
"Withdrawal,-123.45,\"foo bar\t#test\",2024-09-01T12:34:56+08:00,\"Test Account\",\"A expense account\",\"Test Category\"\n"), 0, nil, nil, nil, nil, nil)
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,description,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-123.45,\"foo bar\t#test\",2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\"\n"), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "foo bar\t#test", allNewTransactions[0].Comment)
}
func TestFireFlyIIICsvFileConverterParseImportedData_ParseTags(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, allNewTags, err := converter.ParseImportedData(context, user, []byte("type,amount,tags,date,source_name,source_type,destination_name,destination_type,category\n"+
"Withdrawal,-123.45,\"tag1,tag2,tag3\",2024-09-01T12:34:56+08:00,\"Test Account\",\"Asset account\",\"A expense account\",\"Expense account\",\"Test Category\"\n"), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, 3, len(allNewTags))
assert.Equal(t, int64(1234567890), allNewTags[0].Uid)
assert.Equal(t, "tag1", allNewTags[0].Name)
assert.Equal(t, int64(1234567890), allNewTags[1].Uid)
assert.Equal(t, "tag2", allNewTags[1].Name)
assert.Equal(t, int64(1234567890), allNewTags[2].Uid)
assert.Equal(t, "tag3", allNewTags[2].Name)
}
func TestFireFlyIIICsvFileConverterParseImportedData_MissingFileHeader(t *testing.T) {
converter := FireflyIIITransactionDataCsvFileImporter
context := core.NewNullContext()
@@ -252,7 +437,7 @@ func TestFireFlyIIICsvFileConverterParseImportedData_MissingFileHeader(t *testin
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(""), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
}
func TestFireFlyIIICsvFileConverterParseImportedData_MissingRequiredColumn(t *testing.T) {
@@ -265,32 +450,37 @@ func TestFireFlyIIICsvFileConverterParseImportedData_MissingRequiredColumn(t *te
}
// Missing Time Column
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,source_name,destination_name,category\n"+
"\"Opening balance\",-123.45,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte("type,amount,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Type Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("amount,date,source_name,destination_name,category\n"+
"-123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Sub Category Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name\n"+
"\"Opening balance\",-123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\"\n"), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("amount,date,source_name,source_type,destination_name,destination_type,category\n"+
"123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Account Name Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,destination_name,category\n"+
"\"Opening balance\",-123.45,2024-09-01T00:00:00+08:00,\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
"\"Opening balance\",123.45,2024-09-01T00:00:00+08:00,\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Amount Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,date,source_name,destination_name,category\n"+
"\"Opening balance\",2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,date,source_name,source_type,destination_name,destination_type,category\n"+
"\"Opening balance\",2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Initial balance account\",\"Test Account\",\"Asset account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Account2 Name Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,category\n"+
"\"Opening balance\",-123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\n"), 0, nil, nil, nil, nil, nil)
"\"Opening balance\",123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Source Account Type Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,destination_name,destination_type,category\n"+
"\"Opening balance\",123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Test Account\",\"Asset account\",\"Asset account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
// Missing Destination Account Type Column
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte("type,amount,date,source_name,source_type,destination_name,category\n"+
"\"Opening balance\",123.45,2024-09-01T00:00:00+08:00,\"Initial balance for \"\"Test Account\"\"\",\"Asset account\",\"Test Account\",\n"), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrMissingRequiredFieldInHeaderRow.Message)
}
@@ -1,95 +1,133 @@
package fireflyIII
import (
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
const fireflyIIITransactionTimeColumnName = "date"
const fireflyIIITransactionTypeColumnName = "type"
const fireflyIIITransactionCategoryColumnName = "category"
const fireflyIIITransactionSourceAccountNameColumnName = "source_name"
const fireflyIIITransactionSourceAccountTypeColumnName = "source_type"
const fireflyIIITransactionCurrencyCodeColumnName = "currency_code"
const fireflyIIITransactionAmountColumnName = "amount"
const fireflyIIITransactionDestinationAccountNameColumnName = "destination_name"
const fireflyIIITransactionDestinationAccountTypeColumnName = "destination_type"
const fireflyIIITransactionForeignCurrencyCodeColumnName = "foreign_currency_code"
const fireflyIIITransactionForeignAmountColumnName = "foreign_amount"
const fireflyIIITransactionTagsColumnName = "tags"
const fireflyIIITransactionDescriptionColumnName = "description"
const fireflyIIIAssetAccountName = "Asset account"
const fireflyIIIExpenseAccountName = "Expense account"
const fireflyIIIRevenueAccountName = "Revenue account"
const fireflyIIIDebtAccountName = "Debt"
// fireflyIIITransactionDataRowParser defines the structure of firefly III transaction data row parser
type fireflyIIITransactionDataRowParser struct {
}
// GetAddedColumns returns the added columns after converting the data row
func (p *fireflyIIITransactionDataRowParser) GetAddedColumns() []datatable.TransactionDataTableColumn {
return []datatable.TransactionDataTableColumn{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE,
}
}
// Parse returns the converted transaction data row
func (p *fireflyIIITransactionDataRowParser) Parse(data map[datatable.TransactionDataTableColumn]string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
rowData = make(map[datatable.TransactionDataTableColumn]string, len(data))
rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = ""
for column, value := range data {
rowData[column] = value
}
func (p *fireflyIIITransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataRow datatable.CommonDataTableRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
rowData = make(map[datatable.TransactionDataTableColumn]string, len(fireflyIIITransactionSupportedColumns))
// parse long date time and timezone
if rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] != "" {
if strings.Index(rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME], "T") <= 0 {
return nil, false, errs.ErrTransactionTimeInvalid
}
dateTime, err := utils.ParseFromLongDateTimeWithTimezoneRFC3339Format(dataRow.GetData(fireflyIIITransactionTimeColumnName))
dateTime, err := utils.ParseFromLongDateTimeWithTimezone(strings.ReplaceAll(rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME], "T", " "))
if err != nil {
return nil, false, errs.ErrTransactionTimeInvalid
}
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = utils.FormatUnixTimeToLongDateTime(dateTime.Unix(), dateTime.Location())
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE] = utils.FormatTimezoneOffset(dateTime.Location())
if err != nil {
return nil, false, errs.ErrTransactionTimeInvalid
}
// trim trailing zero in decimal
if rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] != "" {
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.TrimTrailingZerosInDecimal(rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT])
amount, err := utils.ParseAmount(rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT])
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = utils.FormatUnixTimeToLongDateTime(dateTime.Unix(), dateTime.Location())
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIMEZONE] = utils.FormatTimezoneOffset(dateTime.Location())
// parse transaction type, transaction category and amount
transactionType := dataRow.GetData(fireflyIIITransactionTypeColumnName)
sourceAccountType := dataRow.GetData(fireflyIIITransactionSourceAccountTypeColumnName)
destinationAccountType := dataRow.GetData(fireflyIIITransactionDestinationAccountTypeColumnName)
amount, err := utils.ParseAmount(utils.TrimTrailingZerosInDecimal(dataRow.GetData(fireflyIIITransactionAmountColumnName)))
if err != nil {
return nil, false, errs.ErrAmountInvalid
}
foreignAmount := amount
if dataRow.HasData(fireflyIIITransactionForeignAmountColumnName) && dataRow.GetData(fireflyIIITransactionForeignAmountColumnName) != "" {
foreignAmount, err = utils.ParseAmount(utils.TrimTrailingZerosInDecimal(dataRow.GetData(fireflyIIITransactionForeignAmountColumnName)))
if err != nil {
return nil, false, errs.ErrAmountInvalid
}
}
rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(fireflyIIITransactionCategoryColumnName)
if sourceAccountType == fireflyIIIRevenueAccountName && (destinationAccountType == fireflyIIIAssetAccountName || destinationAccountType == fireflyIIIDebtAccountName) { // income
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME]
// if the category is empty, use the source account (revenue account) name as the category name
if rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] == "" {
rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(fireflyIIITransactionSourceAccountNameColumnName)
}
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(fireflyIIITransactionDestinationAccountNameColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
} else if (sourceAccountType == fireflyIIIAssetAccountName || sourceAccountType == fireflyIIIDebtAccountName) && destinationAccountType == fireflyIIIExpenseAccountName { // expense
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE]
// if the category is empty, use the destination account (expense account) name as the category name
if rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] == "" {
rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(fireflyIIITransactionDestinationAccountNameColumnName)
}
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(fireflyIIITransactionSourceAccountNameColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-amount)
}
} else if transactionType == fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_MODIFY_BALANCE] { // opening balance
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_MODIFY_BALANCE]
rowData[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = ""
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(fireflyIIITransactionDestinationAccountNameColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
} else if (sourceAccountType == fireflyIIIAssetAccountName || sourceAccountType == fireflyIIIDebtAccountName) && (destinationAccountType == fireflyIIIAssetAccountName || destinationAccountType == fireflyIIIDebtAccountName) { // transfer
rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER]
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = dataRow.GetData(fireflyIIITransactionSourceAccountNameColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = dataRow.GetData(fireflyIIITransactionDestinationAccountNameColumnName)
if rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] != "" {
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = utils.TrimTrailingZerosInDecimal(rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT])
amount, err := utils.ParseAmount(rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT])
if err != nil {
return nil, false, errs.ErrAmountInvalid
if transactionType == fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE] {
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-amount)
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = utils.FormatAmount(-foreignAmount)
} else {
rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = utils.FormatAmount(foreignAmount)
}
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = utils.FormatAmount(-amount)
} else {
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_AMOUNT] = rowData[datatable.TRANSACTION_DATA_TABLE_AMOUNT]
log.Errorf(ctx, "[fireflyiii_transaction_data_row_parser.Parse] cannot detect transaction type, source account type is \"%s\", destination account type is \"%s\", Firefly III transaction type is \"%s\"", sourceAccountType, destinationAccountType, transactionType)
return nil, false, errs.ErrTransactionTypeInvalid
}
// parse account currency
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = dataRow.GetData(fireflyIIITransactionCurrencyCodeColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY] = dataRow.GetData(fireflyIIITransactionForeignCurrencyCodeColumnName)
// the related account currency field is foreign currency in firefly III actually
if rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY] == "" {
if rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY] == "" && rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] != "" {
rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_CURRENCY] = rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY]
}
// the destination account of modify balance transaction in firefly III is the asset account
if rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] == fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_MODIFY_BALANCE] {
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME]
}
// the destination account of income transaction in firefly III is the asset account
if rowData[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] == fireflyIIITransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] {
rowData[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = rowData[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME]
}
// parse tags / description
rowData[datatable.TRANSACTION_DATA_TABLE_TAGS] = dataRow.GetData(fireflyIIITransactionTagsColumnName)
rowData[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(fireflyIIITransactionDescriptionColumnName)
return rowData, true, nil
}
// createFireflyIIITransactionDataRowParser returns firefly III transaction data row parser
func createFireflyIIITransactionDataRowParser() datatable.TransactionDataRowParser {
func createFireflyIIITransactionDataRowParser() datatable.CommonTransactionDataRowParser {
return &fireflyIIITransactionDataRowParser{}
}
@@ -818,7 +818,7 @@ func TestGnuCashTransactionDatabaseFileParseImportedData_MissingAccountRequiredN
DefaultCurrency: "CNY",
}
// Missing Transaction Time Node
// Missing Account Currency Node
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n"+
"<gnc-v2\n"+
@@ -86,7 +86,7 @@ func (t *gnucashTransactionDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allData)
}
// Next returns the next imported data row
// Next returns the next transaction data row
func (t *gnucashTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
if t.currentIndex+1 >= len(t.dataTable.allData) {
return nil, nil
@@ -177,6 +177,8 @@ func (t *gnucashTransactionDataRowIterator) parseTransaction(ctx core.Context, u
if toAccount.Commodity != nil && toAccount.Commodity.Space == gnucashCommodityCurrencySpace {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = toAccount.Commodity.Id
} else {
return nil, false, errs.ErrAccountCurrencyInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = toAmount
@@ -207,6 +209,8 @@ func (t *gnucashTransactionDataRowIterator) parseTransaction(ctx core.Context, u
if fromAccount.Commodity != nil && fromAccount.Commodity.Space == gnucashCommodityCurrencySpace {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = fromAccount.Commodity.Id
} else {
return nil, false, errs.ErrAccountCurrencyInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = fromAmount
+12 -12
View File
@@ -13,20 +13,20 @@ type iifAccountData struct {
// iifTransactionDataset defines the structure of intuit interchange format (iif) transaction dataset
type iifTransactionDataset struct {
transactionDataColumnIndexes map[string]int
splitDataColumnIndexes map[string]int
transactions []*iifTransactionData
TransactionDataColumnIndexes map[string]int
SplitDataColumnIndexes map[string]int
Transactions []*iifTransactionData
}
// iifTransactionData defines the structure of intuit interchange format (iif) transaction data
type iifTransactionData struct {
dataItems []string
splitData []*iifTransactionSplitData
DataItems []string
SplitData []*iifTransactionSplitData
}
// iifTransactionSplitData defines the structure of intuit interchange format (iif) transaction split data
type iifTransactionSplitData struct {
dataItems []string
DataItems []string
}
func (s *iifTransactionDataset) getTransactionDataItemValue(transactionData *iifTransactionData, columnName string) (string, bool) {
@@ -34,13 +34,13 @@ func (s *iifTransactionDataset) getTransactionDataItemValue(transactionData *iif
return "", false
}
index, exists := s.transactionDataColumnIndexes[columnName]
index, exists := s.TransactionDataColumnIndexes[columnName]
if !exists || index < 0 || index >= len(transactionData.dataItems) {
if !exists || index < 0 || index >= len(transactionData.DataItems) {
return "", false
}
return transactionData.dataItems[index], true
return transactionData.DataItems[index], true
}
func (s *iifTransactionDataset) getSplitDataItemValue(splitData *iifTransactionSplitData, columnName string) (string, bool) {
@@ -48,11 +48,11 @@ func (s *iifTransactionDataset) getSplitDataItemValue(splitData *iifTransactionS
return "", false
}
index, exists := s.splitDataColumnIndexes[columnName]
index, exists := s.SplitDataColumnIndexes[columnName]
if !exists || index < 0 || index >= len(splitData.dataItems) {
if !exists || index < 0 || index >= len(splitData.DataItems) {
return "", false
}
return splitData.dataItems[index], true
return splitData.DataItems[index], true
}
+9 -9
View File
@@ -119,8 +119,8 @@ func (r *iifDataReader) read(ctx core.Context) ([]*iifAccountDataset, []*iifTran
if lastLineSign == "" {
if items[0] == iifTransactionLineSignColumnName {
currentTransactionData = &iifTransactionData{
dataItems: items,
splitData: make([]*iifTransactionSplitData, 0),
DataItems: items,
SplitData: make([]*iifTransactionSplitData, 0),
}
lastLineSign = items[0]
} else {
@@ -134,8 +134,8 @@ func (r *iifDataReader) read(ctx core.Context) ([]*iifAccountDataset, []*iifTran
return nil, nil, errs.ErrInvalidIIFFile
}
currentTransactionData.splitData = append(currentTransactionData.splitData, &iifTransactionSplitData{
dataItems: items,
currentTransactionData.SplitData = append(currentTransactionData.SplitData, &iifTransactionSplitData{
DataItems: items,
})
lastLineSign = items[0]
} else if items[0] == iifTransactionEndLineSignColumnName {
@@ -144,12 +144,12 @@ func (r *iifDataReader) read(ctx core.Context) ([]*iifAccountDataset, []*iifTran
return nil, nil, errs.ErrInvalidIIFFile
}
if len(currentTransactionData.splitData) < 1 {
if len(currentTransactionData.SplitData) < 1 {
log.Errorf(ctx, "[iif_data_reader.read] expected reading transaction split line, but read \"%s\"", items[0])
return nil, nil, errs.ErrInvalidIIFFile
}
currentTransactionDataset.transactions = append(currentTransactionDataset.transactions, currentTransactionData)
currentTransactionDataset.Transactions = append(currentTransactionDataset.Transactions, currentTransactionData)
lastLineSign = ""
} else {
log.Errorf(ctx, "[iif_data_reader.read] iif line expected reading split sign or transaction end sign, but actual is \"%s\"", items[0])
@@ -234,9 +234,9 @@ func (r *iifDataReader) readTransactionSampleLines(ctx core.Context, items []str
}
return &iifTransactionDataset{
transactionDataColumnIndexes: transactionDataColumnIndexes,
splitDataColumnIndexes: splitDataColumnIndexes,
transactions: make([]*iifTransactionData, 0),
TransactionDataColumnIndexes: transactionDataColumnIndexes,
SplitDataColumnIndexes: splitDataColumnIndexes,
Transactions: make([]*iifTransactionData, 0),
}, nil
}
@@ -383,8 +383,8 @@ func TestIIFTransactionDataFileParseImportedData_ParseShortMonthDayTwoDigitsYear
"TRNS\t09/2/24\tTest Account\t123.45\n"+
"SPL\t09/2/24\tTest Account2\t-123.45\n"+
"ENDTRNS\t\t\t\n"+
"TRNS\t9/3/24\tTest Account\t123.45\n"+
"SPL\t9/3/24\tTest Account2\t-123.45\n"+
"TRNS\t24/9/3\tTest Account\t123.45\n"+
"SPL\t24/9/3\tTest Account2\t-123.45\n"+
"ENDTRNS\t\t\t\n"), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
@@ -1,9 +1,7 @@
package iif
import (
"fmt"
"strings"
"time"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
@@ -76,11 +74,11 @@ func (t *iifTransactionDataTable) TransactionRowCount() int {
for i := 0; i < len(t.transactionDatasets); i++ {
datasets := t.transactionDatasets[i]
for j := 0; j < len(datasets.transactions); j++ {
transaction := datasets.transactions[j]
for j := 0; j < len(datasets.Transactions); j++ {
transaction := datasets.Transactions[j]
if transaction.splitData != nil {
totalDataRowCount += len(transaction.splitData)
if transaction.SplitData != nil {
totalDataRowCount += len(transaction.SplitData)
}
}
}
@@ -124,17 +122,17 @@ func (t *iifTransactionDataRowIterator) HasNext() bool {
currentDataset := allDatasets[t.currentDatasetIndex]
if t.currentIndexInDataset+1 < len(currentDataset.transactions) {
if t.currentIndexInDataset+1 < len(currentDataset.Transactions) {
return true
} else if t.currentIndexInDataset < len(currentDataset.transactions) &&
t.currentSplitDataIndex+1 < len(currentDataset.transactions[t.currentIndexInDataset].splitData) {
} else if t.currentIndexInDataset < len(currentDataset.Transactions) &&
t.currentSplitDataIndex+1 < len(currentDataset.Transactions[t.currentIndexInDataset].SplitData) {
return true
}
for i := t.currentDatasetIndex + 1; i < len(allDatasets); i++ {
dataset := allDatasets[i]
if len(dataset.transactions) < 1 {
if len(dataset.Transactions) < 1 {
continue
}
@@ -144,7 +142,7 @@ func (t *iifTransactionDataRowIterator) HasNext() bool {
return false
}
// Next returns the next imported data row
// Next returns the next transaction data row
func (t *iifTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
allDatasets := t.dataTable.transactionDatasets
@@ -152,8 +150,8 @@ func (t *iifTransactionDataRowIterator) Next(ctx core.Context, user *models.User
foundNextRow := false
dataset := allDatasets[i]
for j := t.currentIndexInDataset; j < len(dataset.transactions); j++ {
if t.currentSplitDataIndex+1 < len(dataset.transactions[j].splitData) {
for j := t.currentIndexInDataset; j < len(dataset.Transactions); j++ {
if t.currentSplitDataIndex+1 < len(dataset.Transactions[j].SplitData) {
t.currentSplitDataIndex++
foundNextRow = true
break
@@ -178,22 +176,22 @@ func (t *iifTransactionDataRowIterator) Next(ctx core.Context, user *models.User
currentDataset := allDatasets[t.currentDatasetIndex]
if t.currentIndexInDataset >= len(currentDataset.transactions) {
if t.currentIndexInDataset >= len(currentDataset.Transactions) {
return nil, nil
}
data := currentDataset.transactions[t.currentIndexInDataset]
data := currentDataset.Transactions[t.currentIndexInDataset]
if len(data.splitData) < 1 {
if len(data.SplitData) < 1 {
log.Errorf(ctx, "[iif_transaction_data_table.Next] cannot parsing transaction in row#%d (dataset#%d), because split data is empty", t.currentIndexInDataset, t.currentDatasetIndex)
return nil, errs.ErrInvalidIIFFile
}
if t.currentSplitDataIndex >= len(data.splitData) {
if t.currentSplitDataIndex >= len(data.SplitData) {
return nil, nil
}
if len(data.splitData) > 1 {
if len(data.SplitData) > 1 {
_, err := t.isSplitTransactionSupported(ctx, currentDataset, data)
if err != nil {
@@ -224,11 +222,11 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
return nil, err
}
transactionType, _ := dataset.getSplitDataItemValue(transactionData.splitData[splitDataIndex], iifTransactionTypeColumnName)
transactionType, _ := dataset.getSplitDataItemValue(transactionData.SplitData[splitDataIndex], iifTransactionTypeColumnName)
mainAccountName, _ := dataset.getTransactionDataItemValue(transactionData, iifTransactionAccountNameColumnName)
splitAccountName, _ := dataset.getSplitDataItemValue(transactionData.splitData[splitDataIndex], iifTransactionAccountNameColumnName)
splitAccountName, _ := dataset.getSplitDataItemValue(transactionData.SplitData[splitDataIndex], iifTransactionAccountNameColumnName)
mainAmount, _ := dataset.getTransactionDataItemValue(transactionData, iifTransactionAmountColumnName)
splitAmount, _ := dataset.getSplitDataItemValue(transactionData.splitData[splitDataIndex], iifTransactionAmountColumnName)
splitAmount, _ := dataset.getSplitDataItemValue(transactionData.SplitData[splitDataIndex], iifTransactionAmountColumnName)
mainAmountNum, err := parseAmount(mainAmount)
if err != nil {
@@ -256,7 +254,7 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
categoryName = mainAccountName
accountName = splitAccountName
if len(transactionData.splitData) > 1 {
if len(transactionData.SplitData) > 1 {
amountNum = splitAmountNum
} else {
amountNum = -mainAmountNum
@@ -265,7 +263,7 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
categoryName = splitAccountName
accountName = mainAccountName
if len(transactionData.splitData) > 1 {
if len(transactionData.SplitData) > 1 {
amountNum = -splitAmountNum
} else {
amountNum = mainAmountNum
@@ -297,7 +295,7 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
categoryName = mainAccountName
accountName = splitAccountName
if len(transactionData.splitData) > 1 {
if len(transactionData.SplitData) > 1 {
amountNum = -splitAmountNum
} else {
amountNum = mainAmountNum
@@ -306,7 +304,7 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
categoryName = splitAccountName
accountName = mainAccountName
if len(transactionData.splitData) > 1 {
if len(transactionData.SplitData) > 1 {
amountNum = splitAmountNum
} else {
amountNum = -mainAmountNum
@@ -334,7 +332,7 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
relatedAmountNum := int64(0)
mainAccountTransferToSplitAccount := false
if len(transactionData.splitData) > 1 {
if len(transactionData.SplitData) > 1 {
amountNum = splitAmountNum
relatedAmountNum = splitAmountNum
mainAccountTransferToSplitAccount = amountNum >= 0
@@ -371,11 +369,11 @@ func (t *iifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
}
}
if splitMemo, _ := dataset.getSplitDataItemValue(transactionData.splitData[splitDataIndex], iifTransactionMemoColumnName); splitMemo != "" {
if splitMemo, _ := dataset.getSplitDataItemValue(transactionData.SplitData[splitDataIndex], iifTransactionMemoColumnName); splitMemo != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = splitMemo
} else if memo, _ := dataset.getTransactionDataItemValue(transactionData, iifTransactionMemoColumnName); memo != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = memo
} else if splitName, _ := dataset.getSplitDataItemValue(transactionData.splitData[splitDataIndex], iifTransactionNameColumnName); splitName != "" {
} else if splitName, _ := dataset.getSplitDataItemValue(transactionData.SplitData[splitDataIndex], iifTransactionNameColumnName); splitName != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = splitName
} else if name, _ := dataset.getTransactionDataItemValue(transactionData, iifTransactionNameColumnName); name != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = name
@@ -404,8 +402,8 @@ func (t *iifTransactionDataRowIterator) isSplitTransactionSupported(ctx core.Con
splitTotalAmount := int64(0)
for i := 0; i < len(transactionData.splitData); i++ {
splitAmountStr, _ := dataset.getSplitDataItemValue(transactionData.splitData[i], iifTransactionAmountColumnName)
for i := 0; i < len(transactionData.SplitData); i++ {
splitAmountStr, _ := dataset.getSplitDataItemValue(transactionData.SplitData[i], iifTransactionAmountColumnName)
splitAmount, err := parseAmount(splitAmountStr)
if err != nil {
@@ -422,7 +420,7 @@ func (t *iifTransactionDataRowIterator) isSplitTransactionSupported(ctx core.Con
}
}
if len(transactionData.splitData) > 1 && !supportSplitTransactions {
if len(transactionData.SplitData) > 1 && !supportSplitTransactions {
return false, errs.ErrNotSupportedSplitTransactions
}
@@ -441,25 +439,13 @@ func (t *iifTransactionDataRowIterator) parseTransactionTime(dataset *iifTransac
day := dateParts[1]
year := dateParts[2]
if utils.IsValidYearMonthDayLongOrShortDateFormat(date) {
if utils.IsValidYearMonthDayLongOrShortDateFormat(date) && !utils.IsValidMonthDayYearLongOrShortDateFormat(date) {
year = dateParts[0]
month = dateParts[1]
day = dateParts[2]
}
if len(year) == 2 {
year = utils.IntToString(time.Now().Year()/100) + year
}
if len(month) < 2 {
month = "0" + month
}
if len(day) < 2 {
day = "0" + day
}
return fmt.Sprintf("%s-%s-%s 00:00:00", year, month, day), nil
return utils.FormatYearMonthDayToLongDateTime(year, month, day)
}
func createNewIIfTransactionDataTable(ctx core.Context, accountDatasets []*iifAccountDataset, transactionDatasets []*iifTransactionDataset) (*iifTransactionDataTable, error) {
@@ -477,7 +463,7 @@ func createNewIIfTransactionDataTable(ctx core.Context, accountDatasets []*iifAc
iifTransactionAccountNameColumnName,
iifTransactionAmountColumnName,
} {
if _, exists := transactionDataset.transactionDataColumnIndexes[requiredColumnName]; !exists {
if _, exists := transactionDataset.TransactionDataColumnIndexes[requiredColumnName]; !exists {
return nil, errs.ErrMissingRequiredFieldInHeaderRow
}
}
+73
View File
@@ -0,0 +1,73 @@
package mt
import "strings"
type mtCreditDebitMark string
const (
MT_MARK_CREDIT mtCreditDebitMark = "C"
MT_MARK_DEBIT mtCreditDebitMark = "D"
MT_MARK_REVERSAL_CREDIT mtCreditDebitMark = "RC"
MT_MARK_REVERSAL_DEBIT mtCreditDebitMark = "RD"
)
const (
MT_INFORMATION_TO_ACCOUNT_OWNER_TAG_REMITTANCE string = "REMI"
)
// mt940Data defines the structure of mt940 data
type mt940Data struct {
StatementReferenceNumber string
RelatedReference string
AccountId string
SequentialNumber string
OpeningBalance *mtBalance
ClosingBalance *mtBalance
ClosingAvailableBalance *mtBalance
Statements []*mtStatement
}
// mtStatement defines the structure of mt940 statement
type mtStatement struct {
ValueDate string
EntryDate string
CreditDebitMark mtCreditDebitMark
FundsCode string
Amount string
TransactionTypeIdentificationCode string
ReferenceForAccountOwner string
ReferenceOfAccountServicingInstitution string
InformationToAccountOwner []string
}
// mtBalance defines the structure of mt940 balance
type mtBalance struct {
DebitCreditMark mtCreditDebitMark
Date string
Currency string
Amount string
}
// GetInformationToAccountOwnerMap returns a map of additional information
func (s *mtStatement) GetInformationToAccountOwnerMap() map[string]string {
additionalInfoMap := make(map[string]string, len(s.InformationToAccountOwner))
for _, info := range s.InformationToAccountOwner {
items := strings.Split(info, "/")
if len(items) < 3 {
continue
}
for i := 2; i < len(items); i += 2 {
key := strings.TrimSpace(items[i-1])
value := strings.TrimSpace(items[i])
if len(key) > 0 {
additionalInfoMap[key] = value
}
}
}
return additionalInfoMap
}
+290
View File
@@ -0,0 +1,290 @@
package mt
import (
"bufio"
"bytes"
"strings"
"golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
)
const mtBasicHeaderBlockPrefix = "{1:"
const mtTextBlockStartPrefix = "{4:"
const mtTextBlockEndPrefix = "-}"
const mtTagPrefix = ':'
const mtInformationToAccountOwnerMaxLines = 6
const (
mtTagStatementReferenceNumber = ":20:"
mtTagRelatedReference = ":21:"
mtTagAccountId = ":25:"
mtTagSequentialNumber = ":28C:"
mtTagOpeningBalanceF = ":60F:"
mtTagOpeningBalanceM = ":60M:"
mtTagClosingBalanceF = ":62F:"
mtTagClosingBalanceM = ":62M:"
mtTagClosingAvailableBalance = ":64:"
mtTagStatementLine = ":61:"
mtTagInformationToAccountOwner = ":86:"
)
const (
mtTransactionTypeSwiftTransfer = 'S'
mtTransactionTypeNonSwiftTransfer = 'N'
mtTransactionTypeFirstAdvice = 'F'
)
// mt940DataReader defines the structure of mt940 data reader
type mt940DataReader struct {
allLines []string
}
// read returns the imported mt940 data
// Reference: https://www2.swift.com/knowledgecentre/publications/us9m_20230720/2.0?topic=mt940-format-spec.htm
func (r *mt940DataReader) read(ctx core.Context) (*mt940Data, error) {
if len(r.allLines) < 1 {
return nil, errs.ErrNotFoundTransactionDataInFile
}
data := &mt940Data{}
var currentStatement *mtStatement
var lastTag string
for i := 0; i < len(r.allLines); i++ {
line := strings.TrimSpace(r.allLines[i])
if len(line) < 1 {
continue
}
if strings.HasPrefix(line, mtBasicHeaderBlockPrefix) && strings.HasSuffix(line, mtTextBlockStartPrefix) {
data = &mt940Data{}
currentStatement = nil
lastTag = ""
continue
} else if strings.HasPrefix(line, mtTextBlockEndPrefix) {
break
}
if strings.HasPrefix(line, mtTagStatementReferenceNumber) {
data.StatementReferenceNumber = line[len(mtTagStatementReferenceNumber):]
lastTag = mtTagStatementReferenceNumber
} else if strings.HasPrefix(line, mtTagRelatedReference) {
data.RelatedReference = line[len(mtTagRelatedReference):]
lastTag = mtTagRelatedReference
} else if strings.HasPrefix(line, mtTagAccountId) {
data.AccountId = line[len(mtTagAccountId):]
lastTag = mtTagAccountId
} else if strings.HasPrefix(line, mtTagSequentialNumber) {
data.SequentialNumber = line[len(mtTagSequentialNumber):]
lastTag = mtTagSequentialNumber
} else if strings.HasPrefix(line, mtTagOpeningBalanceF) || strings.HasPrefix(line, mtTagOpeningBalanceM) {
balance, err := r.parseBalance(ctx, line[len(mtTagOpeningBalanceF):])
if err != nil {
return nil, err
}
data.OpeningBalance = balance
lastTag = line[:len(mtTagOpeningBalanceF)]
} else if strings.HasPrefix(line, mtTagClosingBalanceF) || strings.HasPrefix(line, mtTagClosingBalanceM) {
balance, err := r.parseBalance(ctx, line[len(mtTagClosingBalanceF):])
if err != nil {
return nil, err
}
data.ClosingBalance = balance
lastTag = line[:len(mtTagClosingBalanceF)]
} else if strings.HasPrefix(line, mtTagClosingAvailableBalance) {
balance, err := r.parseBalance(ctx, line[len(mtTagClosingAvailableBalance):])
if err != nil {
return nil, err
}
data.ClosingAvailableBalance = balance
lastTag = mtTagClosingAvailableBalance
} else if strings.HasPrefix(line, mtTagStatementLine) {
if currentStatement != nil {
data.Statements = append(data.Statements, currentStatement)
}
statement, err := r.parseStatement(ctx, line[len(mtTagStatementLine):])
if err != nil {
return nil, err
}
currentStatement = statement
lastTag = mtTagStatementLine
} else if strings.HasPrefix(line, mtTagInformationToAccountOwner) && currentStatement != nil {
currentStatement.InformationToAccountOwner = make([]string, 1)
currentStatement.InformationToAccountOwner[0] = line[len(mtTagInformationToAccountOwner):]
lastTag = mtTagInformationToAccountOwner
} else if line[0] != mtTagPrefix && lastTag == mtTagStatementLine && currentStatement != nil {
currentStatement.ReferenceForAccountOwner += line
lastTag = ""
} else if line[0] != mtTagPrefix && lastTag == mtTagInformationToAccountOwner && currentStatement != nil && len(currentStatement.InformationToAccountOwner) < mtInformationToAccountOwnerMaxLines {
currentStatement.InformationToAccountOwner = append(currentStatement.InformationToAccountOwner, line)
lastTag = mtTagInformationToAccountOwner
} else {
log.Warnf(ctx, "[mt_data_reader.read] unsupported line \"%s\" and skip this line", line)
}
}
if currentStatement != nil {
data.Statements = append(data.Statements, currentStatement)
}
return data, nil
}
func (r *mt940DataReader) parseBalance(ctx core.Context, data string) (*mtBalance, error) {
// 1!a (debit/credit mark)
// 6!n (date)
// 3!a (currency)
// 15d (amount)
if len(data) < 9 {
return nil, errs.ErrInvalidMT940File
}
if data[0] != MT_MARK_DEBIT[0] && data[0] != MT_MARK_CREDIT[0] {
log.Errorf(ctx, "[mt_data_reader.parseBalance] cannot parse unknown debit/credit mark, current line is %s", data)
return nil, errs.ErrTransactionTypeInvalid
}
balance := &mtBalance{
DebitCreditMark: mtCreditDebitMark(data[0:1]),
Date: data[1:7],
Currency: data[7:10],
Amount: data[10:],
}
return balance, nil
}
func (r *mt940DataReader) parseStatement(ctx core.Context, data string) (*mtStatement, error) {
// 6!n (value date)
// [4!n] (entry date, optional)
// 2a (debit/credit mark)
// [1!a] (funds code, optional)
// 15d (amount)
// 1!a3!c (transaction type identification code)
// 16x (reference for account owner)
// [//16x] (reference of account servicing institution, optional)
// [34x] (supplementary details, optional)
if len(data) < 6 {
return nil, errs.ErrInvalidMT940File
}
statement := &mtStatement{
ValueDate: data[0:6],
}
currentIndex := 6
// parse entry date if available
if len(data) >= currentIndex+4 && '0' <= data[currentIndex] && data[currentIndex] <= '9' {
statement.EntryDate = data[6:10]
currentIndex += 4
}
// parse debit/credit indicator
if len(data) >= currentIndex+1 && (data[currentIndex] == MT_MARK_DEBIT[0] || data[currentIndex] == MT_MARK_CREDIT[0]) {
statement.CreditDebitMark = mtCreditDebitMark(data[currentIndex])
currentIndex++
} else if len(data) >= currentIndex+2 && (data[currentIndex:currentIndex+2] == string(MT_MARK_REVERSAL_CREDIT) || data[currentIndex:currentIndex+2] == string(MT_MARK_REVERSAL_DEBIT)) {
statement.CreditDebitMark = mtCreditDebitMark(data[currentIndex : currentIndex+2])
currentIndex += 2
} else {
log.Errorf(ctx, "[mt_data_reader.parseStatement] cannot parse unknown debit/credit mark, current line is %s", data)
return nil, errs.ErrTransactionTypeInvalid
}
// parse funds code if available
if len(data) >= currentIndex+1 && ('A' <= data[currentIndex] && data[currentIndex] <= 'Z') {
statement.FundsCode = string(data[currentIndex])
currentIndex++
}
// parse amount
amountValue := ""
for i := currentIndex; i < len(data); i++ {
if len(amountValue) < 15 && ('0' <= data[i] && data[i] <= '9' || data[i] == ',') {
amountValue += string(data[i])
} else {
currentIndex = i
break
}
}
statement.Amount = amountValue
if len(statement.Amount) < 1 {
log.Errorf(ctx, "[mt_data_reader.parseStatement] cannot parse amount, current line is %s", data)
return nil, errs.ErrAmountInvalid
}
// parse transaction type identification code
if len(data) >= currentIndex+4 && (data[currentIndex] == uint8(mtTransactionTypeSwiftTransfer) || data[currentIndex] == uint8(mtTransactionTypeNonSwiftTransfer) || data[currentIndex] == uint8(mtTransactionTypeFirstAdvice)) {
statement.TransactionTypeIdentificationCode = data[currentIndex : currentIndex+4]
currentIndex += 4
} else {
log.Errorf(ctx, "[mt_data_reader.parseStatement] cannot parse transaction type identification code, current line is %s", data)
return nil, errs.ErrInvalidMT940File
}
// parse reference for account owner if available
accountOwnerReference := ""
for i := currentIndex; i < len(data); i++ {
if len(accountOwnerReference) < 16 && (data[i] != '/' || (data[i] == '/' && (i >= len(data)-1 || data[i+1] != '/'))) {
accountOwnerReference += string(data[i])
} else {
currentIndex = i
break
}
}
statement.ReferenceForAccountOwner = accountOwnerReference
if len(statement.ReferenceForAccountOwner) < 1 {
log.Errorf(ctx, "[mt_data_reader.parseStatement] cannot parse reference for account owner, current line is %s", data)
return nil, errs.ErrInvalidMT940File
}
// parse reference of account servicing institution if available
if len(data) >= currentIndex+3 && data[currentIndex] == '/' && data[currentIndex+1] == '/' {
accountServicingInstitutionReference := ""
currentIndex += 2
for i := currentIndex; i < len(data); i++ {
if len(accountServicingInstitutionReference) < 16 {
accountServicingInstitutionReference += string(data[i])
} else {
currentIndex = i
break
}
}
statement.ReferenceOfAccountServicingInstitution = accountServicingInstitutionReference
}
return statement, nil
}
func createNewMT940FileReader(data []byte) *mt940DataReader {
fallback := unicode.UTF8.NewDecoder()
reader := transform.NewReader(bytes.NewReader(data), unicode.BOMOverride(fallback))
scanner := bufio.NewScanner(reader)
allLines := make([]string, 0)
for scanner.Scan() {
allLines = append(allLines, scanner.Text())
}
return &mt940DataReader{
allLines: allLines,
}
}
+341
View File
@@ -0,0 +1,341 @@
package mt
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
func TestMT940DataReaderParse(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
"{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:",
":20:MT940-2025001",
":21:RELATEDREFERENCE",
":25:123456789",
":28C:123/1",
":60F:C250601CNY1234,56",
":61:2506010602DY123,45NTRFTEST//ABC123456",
":86:First Transaction",
"Additional Info",
":61:2506020620CY234,56NSTFFOOBAR//DEF789012",
":86:Second Transaction",
"More Info",
":62F:C250602CNY2345,67",
":64:C250602CNY2345,67",
"-}",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, "MT940-2025001", actualData.StatementReferenceNumber)
assert.Equal(t, "RELATEDREFERENCE", actualData.RelatedReference)
assert.Equal(t, "123456789", actualData.AccountId)
assert.Equal(t, "123/1", actualData.SequentialNumber)
assert.Equal(t, MT_MARK_CREDIT, actualData.OpeningBalance.DebitCreditMark)
assert.Equal(t, "250601", actualData.OpeningBalance.Date)
assert.Equal(t, "CNY", actualData.OpeningBalance.Currency)
assert.Equal(t, "1234,56", actualData.OpeningBalance.Amount)
assert.Equal(t, 2, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, "0602", actualData.Statements[0].EntryDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "Y", actualData.Statements[0].FundsCode)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", actualData.Statements[0].ReferenceForAccountOwner)
assert.Equal(t, "ABC123456", actualData.Statements[0].ReferenceOfAccountServicingInstitution)
assert.Equal(t, "First Transaction", actualData.Statements[0].InformationToAccountOwner[0])
assert.Equal(t, "Additional Info", actualData.Statements[0].InformationToAccountOwner[1])
assert.Equal(t, "250602", actualData.Statements[1].ValueDate)
assert.Equal(t, "0620", actualData.Statements[1].EntryDate)
assert.Equal(t, MT_MARK_CREDIT, actualData.Statements[1].CreditDebitMark)
assert.Equal(t, "Y", actualData.Statements[0].FundsCode)
assert.Equal(t, "234,56", actualData.Statements[1].Amount)
assert.Equal(t, "NSTF", actualData.Statements[1].TransactionTypeIdentificationCode)
assert.Equal(t, "FOOBAR", actualData.Statements[1].ReferenceForAccountOwner)
assert.Equal(t, "DEF789012", actualData.Statements[1].ReferenceOfAccountServicingInstitution)
assert.Equal(t, "Second Transaction", actualData.Statements[1].InformationToAccountOwner[0])
assert.Equal(t, "More Info", actualData.Statements[1].InformationToAccountOwner[1])
assert.Equal(t, MT_MARK_CREDIT, actualData.ClosingBalance.DebitCreditMark)
assert.Equal(t, "250602", actualData.ClosingBalance.Date)
assert.Equal(t, "CNY", actualData.ClosingBalance.Currency)
assert.Equal(t, "2345,67", actualData.ClosingBalance.Amount)
assert.Equal(t, MT_MARK_CREDIT, actualData.ClosingAvailableBalance.DebitCreditMark)
assert.Equal(t, "250602", actualData.ClosingAvailableBalance.Date)
assert.Equal(t, "CNY", actualData.ClosingAvailableBalance.Currency)
assert.Equal(t, "2345,67", actualData.ClosingAvailableBalance.Amount)
}
func TestMT940DataReaderParse_NoBlockHeaderFooter(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
":20:MT940-2025001",
":25:123456789",
":28C:123/1",
":60F:C250601CNY1234,56",
":61:2506010602DY123,45NTRFTEST//ABC123456",
":86:First Transaction",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, "MT940-2025001", actualData.StatementReferenceNumber)
assert.Equal(t, "123456789", actualData.AccountId)
assert.Equal(t, "123/1", actualData.SequentialNumber)
assert.Equal(t, MT_MARK_CREDIT, actualData.OpeningBalance.DebitCreditMark)
assert.Equal(t, "250601", actualData.OpeningBalance.Date)
assert.Equal(t, "CNY", actualData.OpeningBalance.Currency)
assert.Equal(t, "1234,56", actualData.OpeningBalance.Amount)
assert.Equal(t, 1, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, "0602", actualData.Statements[0].EntryDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "Y", actualData.Statements[0].FundsCode)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", actualData.Statements[0].ReferenceForAccountOwner)
assert.Equal(t, "ABC123456", actualData.Statements[0].ReferenceOfAccountServicingInstitution)
assert.Equal(t, "First Transaction", actualData.Statements[0].InformationToAccountOwner[0])
}
func TestMT940DataReaderParse_ReferenceForTheAccountOwnerTwoLine(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
":61:250601D123,45NTRFABCDEFGHIJKLMNOP",
"QRSTUVWXYZ",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "ABCDEFGHIJKLMNOPQRSTUVWXYZ", actualData.Statements[0].ReferenceForAccountOwner)
}
func TestMT940DataReaderParse_InformationToAccountOwnerSixLine(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
":61:250601D123,45NTRFTEST",
":86:Additional Info Line 1",
"Additional Info Line 2",
"Additional Info Line 3",
"Additional Info Line 4",
"Additional Info Line 5",
"Additional Info Line 6",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", actualData.Statements[0].ReferenceForAccountOwner)
assert.Equal(t, 6, len(actualData.Statements[0].InformationToAccountOwner))
assert.Equal(t, "Additional Info Line 1", actualData.Statements[0].InformationToAccountOwner[0])
assert.Equal(t, "Additional Info Line 2", actualData.Statements[0].InformationToAccountOwner[1])
assert.Equal(t, "Additional Info Line 3", actualData.Statements[0].InformationToAccountOwner[2])
assert.Equal(t, "Additional Info Line 4", actualData.Statements[0].InformationToAccountOwner[3])
assert.Equal(t, "Additional Info Line 5", actualData.Statements[0].InformationToAccountOwner[4])
assert.Equal(t, "Additional Info Line 6", actualData.Statements[0].InformationToAccountOwner[5])
}
func TestMT940DataReaderParse_InformationToAccountOwnerMoreThanSixLine(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
":61:250601D123,45NTRFTEST",
":86:Additional Info Line 1",
"Additional Info Line 2",
"Additional Info Line 3",
"Additional Info Line 4",
"Additional Info Line 5",
"Additional Info Line 6",
"Additional Info Line 7",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", actualData.Statements[0].ReferenceForAccountOwner)
assert.Equal(t, 6, len(actualData.Statements[0].InformationToAccountOwner))
assert.Equal(t, "Additional Info Line 1", actualData.Statements[0].InformationToAccountOwner[0])
assert.Equal(t, "Additional Info Line 2", actualData.Statements[0].InformationToAccountOwner[1])
assert.Equal(t, "Additional Info Line 3", actualData.Statements[0].InformationToAccountOwner[2])
assert.Equal(t, "Additional Info Line 4", actualData.Statements[0].InformationToAccountOwner[3])
assert.Equal(t, "Additional Info Line 5", actualData.Statements[0].InformationToAccountOwner[4])
assert.Equal(t, "Additional Info Line 6", actualData.Statements[0].InformationToAccountOwner[5])
}
func TestMT940DataReaderParse_DuplicateBlockHeader(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{
"{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:",
":20:MT940-2025001",
":25:123456789",
":28C:123/1",
":60F:C250601CNY1234,56",
"{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:",
":61:2506010602DY123,45NTRFTEST//ABC123456",
":86:First Transaction",
"-}",
},
}
context := core.NewNullContext()
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, "", actualData.StatementReferenceNumber)
assert.Equal(t, "", actualData.AccountId)
assert.Equal(t, "", actualData.SequentialNumber)
assert.Nil(t, actualData.OpeningBalance)
assert.Equal(t, 1, len(actualData.Statements))
assert.Equal(t, "250601", actualData.Statements[0].ValueDate)
assert.Equal(t, "0602", actualData.Statements[0].EntryDate)
assert.Equal(t, MT_MARK_DEBIT, actualData.Statements[0].CreditDebitMark)
assert.Equal(t, "Y", actualData.Statements[0].FundsCode)
assert.Equal(t, "123,45", actualData.Statements[0].Amount)
assert.Equal(t, "NTRF", actualData.Statements[0].TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", actualData.Statements[0].ReferenceForAccountOwner)
assert.Equal(t, "ABC123456", actualData.Statements[0].ReferenceOfAccountServicingInstitution)
}
func TestMT940DataReaderParse_EmptyContent(t *testing.T) {
reader := &mt940DataReader{
allLines: []string{},
}
context := core.NewNullContext()
_, err := reader.read(context)
assert.EqualError(t, err, errs.ErrNotFoundTransactionDataInFile.Message)
}
func TestMT940DataReaderParseBalance_ValidBalance(t *testing.T) {
reader := &mt940DataReader{}
context := core.NewNullContext()
balance, err := reader.parseBalance(context, "C250601CNY1234,56")
assert.Nil(t, err)
assert.Equal(t, MT_MARK_CREDIT, balance.DebitCreditMark)
assert.Equal(t, "250601", balance.Date)
assert.Equal(t, "CNY", balance.Currency)
assert.Equal(t, "1234,56", balance.Amount)
balance, err = reader.parseBalance(context, "D250602USD2345,67")
assert.Nil(t, err)
assert.Equal(t, MT_MARK_DEBIT, balance.DebitCreditMark)
assert.Equal(t, "250602", balance.Date)
assert.Equal(t, "USD", balance.Currency)
assert.Equal(t, "2345,67", balance.Amount)
}
func TestMT940DataReaderParseBalance_InvalidBalance(t *testing.T) {
reader := &mt940DataReader{}
context := core.NewNullContext()
_, err := reader.parseBalance(context, "X250601CNY1234,56")
assert.EqualError(t, err, errs.ErrTransactionTypeInvalid.Message)
_, err = reader.parseBalance(context, "C")
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
}
func TestMT940DataReaderParseStatement_ValidFields(t *testing.T) {
reader := &mt940DataReader{}
context := core.NewNullContext()
statement, err := reader.parseStatement(context, "2506010602RDY123,45NTRFTEST//ABC123456")
assert.Nil(t, err)
assert.Equal(t, "250601", statement.ValueDate)
assert.Equal(t, "0602", statement.EntryDate)
assert.Equal(t, MT_MARK_REVERSAL_DEBIT, statement.CreditDebitMark)
assert.Equal(t, "Y", statement.FundsCode)
assert.Equal(t, "123,45", statement.Amount)
assert.Equal(t, "NTRF", statement.TransactionTypeIdentificationCode)
assert.Equal(t, "TEST", statement.ReferenceForAccountOwner)
assert.Equal(t, "ABC123456", statement.ReferenceOfAccountServicingInstitution)
statement, err = reader.parseStatement(context, "250601RC234,56NSTFFOOBAR")
assert.Nil(t, err)
assert.Equal(t, "250601", statement.ValueDate)
assert.Equal(t, "", statement.EntryDate)
assert.Equal(t, MT_MARK_REVERSAL_CREDIT, statement.CreditDebitMark)
assert.Equal(t, "234,56", statement.Amount)
assert.Equal(t, "NSTF", statement.TransactionTypeIdentificationCode)
assert.Equal(t, "FOOBAR", statement.ReferenceForAccountOwner)
}
func TestMT940DataReaderParseStatement_InvalidField(t *testing.T) {
reader := &mt940DataReader{}
context := core.NewNullContext()
_, err := reader.parseStatement(context, "250601X123,45NTRFTest")
assert.EqualError(t, err, errs.ErrTransactionTypeInvalid.Message)
}
func TestMT940DataReaderParseStatement_MissingField(t *testing.T) {
reader := &mt940DataReader{}
context := core.NewNullContext()
// Missing entry date
_, err := reader.parseStatement(context, "2406")
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
// Missing debit/credit mark
_, err = reader.parseStatement(context, "250601060234,56NTRFTEST//ABC123456")
assert.EqualError(t, err, errs.ErrTransactionTypeInvalid.Message)
// Missing amount
_, err = reader.parseStatement(context, "250601DNTRFTEST//ABC123456")
assert.EqualError(t, err, errs.ErrAmountInvalid.Message)
// Missing transaction type identification code
_, err = reader.parseStatement(context, "250601D234,56TEST//ABC123456")
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
// Missing reference for account owner
_, err = reader.parseStatement(context, "250601D234,56NTRF//ABC123456")
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
}
+91
View File
@@ -0,0 +1,91 @@
package mt
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestMtStatementGetInformationToAccountOwnerMap_OneLineMultiTags(t *testing.T) {
statement := &mtStatement{
InformationToAccountOwner: []string{
"/REMI/test value/ABC/123/FOO/Bar",
},
}
expectedMap := map[string]string{
"REMI": "test value",
"ABC": "123",
"FOO": "Bar",
}
actualMap := statement.GetInformationToAccountOwnerMap()
assert.Equal(t, expectedMap, actualMap)
}
func TestMtStatementGetInformationToAccountOwnerMap_MultipleLines(t *testing.T) {
statement := &mtStatement{
InformationToAccountOwner: []string{
"/REMI/test/ABC/123",
"/FOO/Bar/HELLO/World",
},
}
expectedMap := map[string]string{
"REMI": "test",
"ABC": "123",
"FOO": "Bar",
"HELLO": "World",
}
actualMap := statement.GetInformationToAccountOwnerMap()
assert.Equal(t, expectedMap, actualMap)
}
func TestMtStatementGetInformationToAccountOwnerMap_EmptyInformation(t *testing.T) {
statement := &mtStatement{
InformationToAccountOwner: []string{},
}
expectedMap := map[string]string{}
actualMap := statement.GetInformationToAccountOwnerMap()
assert.Equal(t, expectedMap, actualMap)
}
func TestMtStatementGetInformationToAccountOwnerMap_InvalidFormat(t *testing.T) {
statement := &mtStatement{
InformationToAccountOwner: []string{
"/ABCD",
"EFGH/123",
"/REMI/123/ABC",
},
}
expectedMap := map[string]string{
"REMI": "123",
}
actualMap := statement.GetInformationToAccountOwnerMap()
assert.Equal(t, expectedMap, actualMap)
}
func TestMtStatementGetInformationToAccountOwnerMap_EmptyKeyValue(t *testing.T) {
statement := &mtStatement{
InformationToAccountOwner: []string{
"/REMI//ABC/ /DEF/456",
"/GHI/123/JKL/def",
},
}
expectedMap := map[string]string{
"REMI": "",
"ABC": "",
"DEF": "456",
"GHI": "123",
"JKL": "def",
}
actualMap := statement.GetInformationToAccountOwnerMap()
assert.Equal(t, expectedMap, actualMap)
}
@@ -0,0 +1,42 @@
package mt
import (
"github.com/mayswind/ezbookkeeping/pkg/converters/converter"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
var mt940TransactionTypeNameMapping = map[models.TransactionType]string{
models.TRANSACTION_TYPE_INCOME: utils.IntToString(int(models.TRANSACTION_TYPE_INCOME)),
models.TRANSACTION_TYPE_EXPENSE: utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE)),
models.TRANSACTION_TYPE_TRANSFER: utils.IntToString(int(models.TRANSACTION_TYPE_TRANSFER)),
}
// mt940TransactionDataFileImporter defines the structure of mt940 file importer for statement data
type mt940TransactionDataFileImporter struct{}
// Initialize a mt940 statement data importer singleton instance
var (
MT940TransactionDataFileImporter = &mt940TransactionDataFileImporter{}
)
// ParseImportedData returns the imported data by parsing the mt940 file statement data
func (c *mt940TransactionDataFileImporter) ParseImportedData(ctx core.Context, user *models.User, data []byte, defaultTimezoneOffset int16, accountMap map[string]*models.Account, expenseCategoryMap map[string]map[string]*models.TransactionCategory, incomeCategoryMap map[string]map[string]*models.TransactionCategory, transferCategoryMap map[string]map[string]*models.TransactionCategory, tagMap map[string]*models.TransactionTag) (models.ImportedTransactionSlice, []*models.Account, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionCategory, []*models.TransactionTag, error) {
mt940DataReader := createNewMT940FileReader(data)
mt940Data, err := mt940DataReader.read(ctx)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
transactionDataTable, err := createNewMT940TransactionDataTable(mt940Data)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(mt940TransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
@@ -0,0 +1,214 @@
package mt
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
func TestMT940TransactionDataFileParseImportedData_MinimumValidData(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, allNewAccounts, allNewSubExpenseCategories, allNewSubIncomeCategories, allNewSubTransferCategories, allNewTags, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:2506010602C123,45NTRFTEST//ABC123456
:86:Transaction 1
:61:2506020603D234,56NTRFFOOBAR
:86:Transaction 2
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 2, len(allNewTransactions))
assert.Equal(t, 1, len(allNewAccounts))
assert.Equal(t, 1, len(allNewSubExpenseCategories))
assert.Equal(t, 1, len(allNewSubIncomeCategories))
assert.Equal(t, 0, len(allNewSubTransferCategories))
assert.Equal(t, 0, len(allNewTags))
assert.Equal(t, int64(1234567890), allNewTransactions[0].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, int64(1748736000), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
assert.Equal(t, int64(12345), allNewTransactions[0].Amount)
assert.Equal(t, "12345678", allNewTransactions[0].OriginalSourceAccountName)
assert.Equal(t, "CNY", allNewTransactions[0].OriginalSourceAccountCurrency)
assert.Equal(t, "", allNewTransactions[0].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewTransactions[1].Uid)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[1].Type)
assert.Equal(t, int64(1748822400), utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime))
assert.Equal(t, int64(23456), allNewTransactions[1].Amount)
assert.Equal(t, "12345678", allNewTransactions[1].OriginalSourceAccountName)
assert.Equal(t, "CNY", allNewTransactions[1].OriginalSourceAccountCurrency)
assert.Equal(t, "", allNewTransactions[1].OriginalCategoryName)
assert.Equal(t, int64(1234567890), allNewAccounts[0].Uid)
assert.Equal(t, "12345678", allNewAccounts[0].Name)
assert.Equal(t, "CNY", allNewAccounts[0].Currency)
assert.Equal(t, int64(1234567890), allNewSubExpenseCategories[0].Uid)
assert.Equal(t, "", allNewSubExpenseCategories[0].Name)
assert.Equal(t, int64(1234567890), allNewSubIncomeCategories[0].Uid)
assert.Equal(t, "", allNewSubIncomeCategories[0].Name)
}
func TestMT940TransactionDataFileParseImportedData_ParseTransactionValidAmountAndCurrency(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:250601C123,45NTRFTEST
:86:Transaction 1
:61:250602C0,12NTRFTEST
:86:Transaction 2
:61:250603C1,NTRFTEST
:86:Transaction 3
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 3, len(allNewTransactions))
assert.Equal(t, int64(12345), allNewTransactions[0].Amount)
assert.Equal(t, int64(12), allNewTransactions[1].Amount)
assert.Equal(t, int64(100), allNewTransactions[2].Amount)
}
func TestMT940TransactionDataFileParseImportedData_ParseTransactionInvalidAmountAndCurrency(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:2506010602C123 45NTRFTEST
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
_, _, _, _, _, _, err = converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:2506010602C12.45NTRFTEST
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrInvalidMT940File.Message)
}
func TestMT940TransactionDataFileParseImportedData_ParseTransactionType(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:250601C123,45NTRFTEST
:86:Transaction 1
:61:250602D123,45NTRFTEST
:86:Transaction 2
:61:250603RC123,45NTRFTEST
:86:Transaction 3
:61:250604RD123,45NTRFTEST
:86:Transaction 4
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 4, len(allNewTransactions))
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[0].Type)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[1].Type)
assert.Equal(t, models.TRANSACTION_DB_TYPE_EXPENSE, allNewTransactions[2].Type)
assert.Equal(t, models.TRANSACTION_DB_TYPE_INCOME, allNewTransactions[3].Type)
}
func TestMT940TransactionDataFileParseImportedData_ParseDescription(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:25:12345678
:28C:123/1
:60F:C250601CNY123,45
:61:2506010602C123,45NTRFTEST
:86:Transaction 1
Part 2
Part 3
:62F:C250601CNY123,45
-}`), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 1, len(allNewTransactions))
assert.Equal(t, "Transaction 1\nPart 2\nPart 3", allNewTransactions[0].Comment)
}
func TestMT940TransactionDataFileParseImportedData_MissingRequiredField(t *testing.T) {
converter := MT940TransactionDataFileImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
// Missing opening balance and closing balance
_, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
`{1:F01TESTBANK123456789}{2:I940TESTBANK}{4:
:20:123456789
:28C:123/1
:61:250601C123,45NTRFTEST
:86:Transaction 1
-}`), 0, nil, nil, nil, nil, nil)
assert.EqualError(t, err, errs.ErrAccountCurrencyInvalid.Message)
}
@@ -0,0 +1,178 @@
package mt
import (
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/core"
"github.com/mayswind/ezbookkeeping/pkg/errs"
"github.com/mayswind/ezbookkeeping/pkg/log"
"github.com/mayswind/ezbookkeeping/pkg/models"
"github.com/mayswind/ezbookkeeping/pkg/utils"
)
var mt940TransactionSupportedColumns = map[datatable.TransactionDataTableColumn]bool{
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME: true,
datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE: true,
datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY: true,
datatable.TRANSACTION_DATA_TABLE_AMOUNT: true,
datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME: true,
datatable.TRANSACTION_DATA_TABLE_DESCRIPTION: true,
}
// mt940TransactionDataTable represents the mt940 statement data dataTable
type mt940TransactionDataTable struct {
data *mt940Data
}
// mt940TransactionDataRow represents a row in the mt940 statement data dataTable
type mt940TransactionDataRow struct {
statement *mtStatement
finalItems map[datatable.TransactionDataTableColumn]string
}
// mt940TransactionDataRowIterator represents an iterator for mt940 statement data rows
type mt940TransactionDataRowIterator struct {
dataTable *mt940TransactionDataTable
currentIndex int
}
// HasColumn implements TransactionDataTable.HasColumn
func (t *mt940TransactionDataTable) HasColumn(column datatable.TransactionDataTableColumn) bool {
_, exists := mt940TransactionSupportedColumns[column]
return exists
}
// TransactionRowCount implements TransactionDataTable.TransactionRowCount
func (t *mt940TransactionDataTable) TransactionRowCount() int {
return len(t.data.Statements)
}
// TransactionRowIterator implements TransactionDataTable.TransactionRowIterator
func (t *mt940TransactionDataTable) TransactionRowIterator() datatable.TransactionDataRowIterator {
return &mt940TransactionDataRowIterator{
dataTable: t,
currentIndex: -1,
}
}
// IsValid implements TransactionDataRow.IsValid
func (r *mt940TransactionDataRow) IsValid() bool {
return true
}
// GetData implements TransactionDataRow.GetData
func (r *mt940TransactionDataRow) GetData(column datatable.TransactionDataTableColumn) string {
_, exists := mt940TransactionSupportedColumns[column]
if exists {
return r.finalItems[column]
}
return ""
}
// HasNext implements TransactionDataRowIterator.HasNext
func (t *mt940TransactionDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.data.Statements)
}
// Next implements TransactionDataRowIterator.Next
func (t *mt940TransactionDataRowIterator) Next(ctx core.Context, user *models.User) (datatable.TransactionDataRow, error) {
if t.currentIndex+1 >= len(t.dataTable.data.Statements) {
return nil, nil
}
t.currentIndex++
data := t.dataTable.data.Statements[t.currentIndex]
rowItems, err := t.parseTransaction(ctx, user, t.dataTable.data, data)
if err != nil {
log.Errorf(ctx, "[mt_transaction_data_table.Next] cannot parsing transaction in row#%d, because %s", t.currentIndex, err.Error())
return nil, err
}
return &mt940TransactionDataRow{
statement: data,
finalItems: rowItems,
}, nil
}
func (t *mt940TransactionDataRowIterator) parseTransaction(ctx core.Context, user *models.User, mt940Data *mt940Data, statement *mtStatement) (map[datatable.TransactionDataTableColumn]string, error) {
data := make(map[datatable.TransactionDataTableColumn]string, len(mt940TransactionSupportedColumns))
if statement.ValueDate == "" && len(statement.ValueDate) != 6 {
return nil, errs.ErrTransactionTimeInvalid
}
transactionTime, err := utils.FormatYearMonthDayToLongDateTime(statement.ValueDate[0:2], statement.ValueDate[2:4], statement.ValueDate[4:6])
if err != nil {
log.Errorf(ctx, "[mt_transaction_data_table.parseTransaction] cannot format transaction time in row#%d, because %s", t.currentIndex, err.Error())
return nil, errs.ErrTransactionTimeInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = transactionTime
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = mt940Data.AccountId
if mt940Data.OpeningBalance != nil && mt940Data.OpeningBalance.Currency != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = mt940Data.OpeningBalance.Currency
} else if mt940Data.ClosingBalance != nil && mt940Data.ClosingBalance.Currency != "" {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = mt940Data.ClosingBalance.Currency
} else {
return nil, errs.ErrAccountCurrencyInvalid
}
amountValue := strings.ReplaceAll(statement.Amount, ",", ".") // decimal separator is comma in mt data
if len(amountValue) > 0 && amountValue[len(amountValue)-1] == '.' {
amountValue = amountValue[:len(amountValue)-1]
}
amount, err := utils.ParseAmount(amountValue)
if err != nil {
log.Errorf(ctx, "[mt_transaction_data_table.parseTransaction] cannot parsing transaction amount \"%s\", because %s", statement.Amount, err.Error())
return nil, errs.ErrAmountInvalid
}
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
if statement.CreditDebitMark == MT_MARK_CREDIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_INCOME))
} else if statement.CreditDebitMark == MT_MARK_DEBIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE))
} else if statement.CreditDebitMark == MT_MARK_REVERSAL_CREDIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_EXPENSE))
} else if statement.CreditDebitMark == MT_MARK_REVERSAL_DEBIT {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = utils.IntToString(int(models.TRANSACTION_TYPE_INCOME))
} else {
return nil, errs.ErrTransactionTypeInvalid
}
informationToAccountOwnerMap := statement.GetInformationToAccountOwnerMap()
if len(informationToAccountOwnerMap) > 0 {
if value, exists := informationToAccountOwnerMap[MT_INFORMATION_TO_ACCOUNT_OWNER_TAG_REMITTANCE]; exists {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = value
}
} else {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = strings.Join(statement.InformationToAccountOwner, "\n")
}
return data, nil
}
// createNewMT940TransactionDataTable creates a new mt940 statement data dataTable
func createNewMT940TransactionDataTable(data *mt940Data) (*mt940TransactionDataTable, error) {
if data == nil || len(data.Statements) < 1 {
return nil, errs.ErrNotFoundTransactionDataInFile
}
return &mt940TransactionDataTable{
data: data,
}, nil
}
+1
View File
@@ -47,6 +47,7 @@ type ofxVersion2FileReader struct {
}
// read returns the imported open financial exchange (ofx) file
// Reference: https://www.financialdataexchange.org/FDX/FDX/About/OFX-Work-Group.aspx?a315d1c24e44=2
func (r *ofxVersion1FileReader) read(ctx core.Context) (*ofxFile, error) {
file := &ofxFile{}
@@ -93,7 +93,7 @@ func (t *ofxTransactionDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allData)
}
// Next returns the next imported data row
// Next returns the next transaction data row
func (t *ofxTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
if t.currentIndex+1 >= len(t.dataTable.allData) {
return nil, nil
@@ -148,6 +148,10 @@ func (t *ofxTransactionDataRowIterator) parseTransaction(ctx core.Context, user
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] = ofxTransaction.DefaultCurrency
}
if data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_CURRENCY] == "" {
return nil, errs.ErrAccountCurrencyInvalid
}
if ofxTransaction.Amount == "" {
return nil, errs.ErrAmountInvalid
}
+58 -58
View File
@@ -34,91 +34,91 @@ const (
// qifData defines the structure of quicken interchange format (qif) data
type qifData struct {
bankAccountTransactions []*qifTransactionData
cashAccountTransactions []*qifTransactionData
creditCardAccountTransactions []*qifTransactionData
assetAccountTransactions []*qifTransactionData
liabilityAccountTransactions []*qifTransactionData
memorizedTransactions []*qifMemorizedTransactionData
investmentAccountTransactions []*qifInvestmentTransactionData
accounts []*qifAccountData
categories []*qifCategoryData
classes []*qifClassData
BankAccountTransactions []*qifTransactionData
CashAccountTransactions []*qifTransactionData
CreditCardAccountTransactions []*qifTransactionData
AssetAccountTransactions []*qifTransactionData
LiabilityAccountTransactions []*qifTransactionData
MemorizedTransactions []*qifMemorizedTransactionData
InvestmentAccountTransactions []*qifInvestmentTransactionData
Accounts []*qifAccountData
Categories []*qifCategoryData
Classes []*qifClassData
}
// qifTransactionData defines the structure of quicken interchange format (qif) transaction data
type qifTransactionData struct {
date string
amount string
clearedStatus qifTransactionClearedStatus
num string
payee string
memo string
addresses []string
category string
subTransactionCategory []string
subTransactionMemo []string
subTransactionAmount []string
account *qifAccountData
Date string
Amount string
ClearedStatus qifTransactionClearedStatus
Num string
Payee string
Memo string
Addresses []string
Category string
SubTransactionCategory []string
SubTransactionMemo []string
SubTransactionAmount []string
Account *qifAccountData
}
// qifInvestmentTransactionData defines the structure of quicken interchange format (qif) investment transaction data
type qifInvestmentTransactionData struct {
date string
action string
security string
price string
quantity string
amount string
clearedStatus qifTransactionClearedStatus
text string
memo string
commission string
accountForTransfer string
amountTransferred string
account *qifAccountData
Date string
Action string
Security string
Price string
Quantity string
Amount string
ClearedStatus qifTransactionClearedStatus
Text string
Memo string
Commission string
AccountForTransfer string
AmountTransferred string
Account *qifAccountData
}
// qifMemorizedTransactionData defines the structure of quicken interchange format (qif) memorized transaction data
type qifMemorizedTransactionData struct {
qifTransactionData
transactionType qifTransactionType
amortization qifMemorizedTransactionAmortizationData
TransactionType qifTransactionType
Amortization qifMemorizedTransactionAmortizationData
}
// qifMemorizedTransactionAmortizationData defines the structure of quicken interchange format (qif) memorized transaction amortization data
type qifMemorizedTransactionAmortizationData struct {
firstPaymentDate string
totalYearsForLoan string
numberOfPayments string
numberOfPeriodsPerYear string
interestRate string
currentLoanBalance string
originalLoanAmount string
FirstPaymentDate string
TotalYearsForLoan string
NumberOfPayments string
NumberOfPeriodsPerYear string
InterestRate string
CurrentLoanBalance string
OriginalLoanAmount string
}
// qifAccountData defines the structure of quicken interchange format (qif) account data
type qifAccountData struct {
name string
accountType string
description string
creditLimit string
statementBalanceDate string
statementBalanceAmount string
Name string
AccountType string
Description string
CreditLimit string
StatementBalanceDate string
StatementBalanceAmount string
}
// qifCategoryData defines the structure of quicken interchange format (qif) category data
type qifCategoryData struct {
name string
description string
taxRelated bool
categoryType qifCategoryType
budgetAmount string
taxScheduleInformation string
Name string
Description string
TaxRelated bool
CategoryType qifCategoryType
BudgetAmount string
TaxScheduleInformation string
}
// qifClassData defines the structure of quicken interchange format (qif) class data
type qifClassData struct {
name string
description string
Name string
Description string
}
+66 -66
View File
@@ -98,18 +98,18 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
continue
}
transactionData.account = currentAccount
transactionData.Account = currentAccount
if currentEntryHeader == qifBankTransactionHeader {
data.bankAccountTransactions = append(data.bankAccountTransactions, transactionData)
data.BankAccountTransactions = append(data.BankAccountTransactions, transactionData)
} else if currentEntryHeader == qifCashTransactionHeader {
data.cashAccountTransactions = append(data.cashAccountTransactions, transactionData)
data.CashAccountTransactions = append(data.CashAccountTransactions, transactionData)
} else if currentEntryHeader == qifCreditCardTransactionHeader {
data.creditCardAccountTransactions = append(data.creditCardAccountTransactions, transactionData)
data.CreditCardAccountTransactions = append(data.CreditCardAccountTransactions, transactionData)
} else if currentEntryHeader == qifAssetAccountTransactionHeader {
data.assetAccountTransactions = append(data.assetAccountTransactions, transactionData)
data.AssetAccountTransactions = append(data.AssetAccountTransactions, transactionData)
} else if currentEntryHeader == qifLiabilityAccountTransactionHeader {
data.liabilityAccountTransactions = append(data.liabilityAccountTransactions, transactionData)
data.LiabilityAccountTransactions = append(data.LiabilityAccountTransactions, transactionData)
}
} else if currentEntryHeader == qifMemorizedTransactionHeader || currentEntryHeader == qifMemorisedTransactionHeader {
transactionData, err := r.parseMemorizedTransaction(ctx, entryData)
@@ -122,8 +122,8 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
continue
}
transactionData.account = currentAccount
data.memorizedTransactions = append(data.memorizedTransactions, transactionData)
transactionData.Account = currentAccount
data.MemorizedTransactions = append(data.MemorizedTransactions, transactionData)
} else if currentEntryHeader == qifInvestmentTransactionHeader {
transactionData, err := r.parseInvestmentTransaction(ctx, entryData)
@@ -135,8 +135,8 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
continue
}
transactionData.account = currentAccount
data.investmentAccountTransactions = append(data.investmentAccountTransactions, transactionData)
transactionData.Account = currentAccount
data.InvestmentAccountTransactions = append(data.InvestmentAccountTransactions, transactionData)
} else if currentEntryHeader == qifAccountHeader {
accountData, err := r.parseAccount(ctx, entryData)
@@ -149,7 +149,7 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
}
currentAccount = accountData
data.accounts = append(data.accounts, accountData)
data.Accounts = append(data.Accounts, accountData)
} else if currentEntryHeader == qifCategoryHeader {
categoryData, err := r.parseCategory(ctx, entryData)
@@ -161,7 +161,7 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
continue
}
data.categories = append(data.categories, categoryData)
data.Categories = append(data.Categories, categoryData)
} else if currentEntryHeader == qifClassHeader {
classData, err := r.parseClass(ctx, entryData)
@@ -173,7 +173,7 @@ func (r *qifDataReader) read(ctx core.Context) (*qifData, error) {
continue
}
data.classes = append(data.classes, classData)
data.Classes = append(data.Classes, classData)
} else {
log.Warnf(ctx, "[qif_data_reader.read] read unsupported entry header \"%s\" and skip this entry", currentEntryHeader)
}
@@ -202,27 +202,27 @@ func (r *qifDataReader) parseTransaction(ctx core.Context, data []string, ignore
}
if line[0] == 'D' {
transactionData.date = line[1:]
transactionData.Date = line[1:]
} else if line[0] == 'T' {
transactionData.amount = line[1:]
transactionData.Amount = line[1:]
} else if line[0] == 'C' {
transactionData.clearedStatus = r.parseClearedStatus(ctx, line[1:])
transactionData.ClearedStatus = r.parseClearedStatus(ctx, line[1:])
} else if line[0] == 'N' {
transactionData.num = line[1:]
transactionData.Num = line[1:]
} else if line[0] == 'P' {
transactionData.payee = line[1:]
transactionData.Payee = line[1:]
} else if line[0] == 'M' {
transactionData.memo = line[1:]
transactionData.Memo = line[1:]
} else if line[0] == 'A' {
transactionData.addresses = append(transactionData.addresses, line[1:])
transactionData.Addresses = append(transactionData.Addresses, line[1:])
} else if line[0] == 'L' {
transactionData.category = line[1:]
transactionData.Category = line[1:]
} else if line[0] == 'S' {
transactionData.subTransactionCategory = append(transactionData.subTransactionCategory, line[1:])
transactionData.SubTransactionCategory = append(transactionData.SubTransactionCategory, line[1:])
} else if line[0] == 'E' {
transactionData.subTransactionMemo = append(transactionData.subTransactionMemo, line[1:])
transactionData.SubTransactionMemo = append(transactionData.SubTransactionMemo, line[1:])
} else if line[0] == '$' {
transactionData.subTransactionAmount = append(transactionData.subTransactionAmount, line[1:])
transactionData.SubTransactionAmount = append(transactionData.SubTransactionAmount, line[1:])
} else {
if !ignoreUnknown {
log.Warnf(ctx, "[qif_data_reader.parseTransaction] read unsupported line \"%s\" and skip this line", line)
@@ -247,7 +247,7 @@ func (r *qifDataReader) parseMemorizedTransaction(ctx core.Context, data []strin
transactionData := &qifMemorizedTransactionData{
qifTransactionData: *baseTransactionData,
amortization: qifMemorizedTransactionAmortizationData{},
Amortization: qifMemorizedTransactionAmortizationData{},
}
for i := 0; i < len(data); i++ {
@@ -266,33 +266,33 @@ func (r *qifDataReader) parseMemorizedTransaction(ctx core.Context, data []strin
if line[0] == 'K' {
if line == string(qifCheckTransactionType) {
transactionData.transactionType = qifCheckTransactionType
transactionData.TransactionType = qifCheckTransactionType
} else if line == string(qifDepositTransactionType) {
transactionData.transactionType = qifDepositTransactionType
transactionData.TransactionType = qifDepositTransactionType
} else if line == string(qifPaymentTransactionType) {
transactionData.transactionType = qifPaymentTransactionType
transactionData.TransactionType = qifPaymentTransactionType
} else if line == string(qifInvestmentTransactionType) {
transactionData.transactionType = qifInvestmentTransactionType
transactionData.TransactionType = qifInvestmentTransactionType
} else if line == string(qifElectronicPayeeTransactionType) {
transactionData.transactionType = qifElectronicPayeeTransactionType
transactionData.TransactionType = qifElectronicPayeeTransactionType
} else {
log.Warnf(ctx, "[qif_data_reader.parseMemorizedTransaction] read unsupported transaction type \"%s\" and skip this line", line)
continue
}
} else if line[0] == '1' {
transactionData.amortization.firstPaymentDate = line[1:]
transactionData.Amortization.FirstPaymentDate = line[1:]
} else if line[0] == '2' {
transactionData.amortization.totalYearsForLoan = line[1:]
transactionData.Amortization.TotalYearsForLoan = line[1:]
} else if line[0] == '3' {
transactionData.amortization.numberOfPayments = line[1:]
transactionData.Amortization.NumberOfPayments = line[1:]
} else if line[0] == '4' {
transactionData.amortization.numberOfPeriodsPerYear = line[1:]
transactionData.Amortization.NumberOfPeriodsPerYear = line[1:]
} else if line[0] == '5' {
transactionData.amortization.interestRate = line[1:]
transactionData.Amortization.InterestRate = line[1:]
} else if line[0] == '6' {
transactionData.amortization.currentLoanBalance = line[1:]
transactionData.Amortization.CurrentLoanBalance = line[1:]
} else if line[0] == '7' {
transactionData.amortization.originalLoanAmount = line[1:]
transactionData.Amortization.OriginalLoanAmount = line[1:]
} else {
log.Warnf(ctx, "[qif_data_reader.parseMemorizedTransaction] read unsupported line \"%s\" and skip this line", line)
continue
@@ -317,29 +317,29 @@ func (r *qifDataReader) parseInvestmentTransaction(ctx core.Context, data []stri
}
if line[0] == 'D' {
transactionData.date = line[1:]
transactionData.Date = line[1:]
} else if line[0] == 'N' {
transactionData.action = line[1:]
transactionData.Action = line[1:]
} else if line[0] == 'Y' {
transactionData.security = line[1:]
transactionData.Security = line[1:]
} else if line[0] == 'I' {
transactionData.price = line[1:]
transactionData.Price = line[1:]
} else if line[0] == 'Q' {
transactionData.quantity = line[1:]
transactionData.Quantity = line[1:]
} else if line[0] == 'T' {
transactionData.amount = line[1:]
transactionData.Amount = line[1:]
} else if line[0] == 'C' {
transactionData.clearedStatus = r.parseClearedStatus(ctx, line[1:])
transactionData.ClearedStatus = r.parseClearedStatus(ctx, line[1:])
} else if line[0] == 'P' {
transactionData.text = line[1:]
transactionData.Text = line[1:]
} else if line[0] == 'M' {
transactionData.memo = line[1:]
transactionData.Memo = line[1:]
} else if line[0] == 'O' {
transactionData.commission = line[1:]
transactionData.Commission = line[1:]
} else if line[0] == 'L' {
transactionData.accountForTransfer = line[1:]
transactionData.AccountForTransfer = line[1:]
} else if line[0] == '$' {
transactionData.amountTransferred = line[1:]
transactionData.AmountTransferred = line[1:]
} else {
log.Warnf(ctx, "[qif_data_reader.parseInvestmentTransaction] read unsupported line \"%s\" and skip this line", line)
continue
@@ -364,17 +364,17 @@ func (r *qifDataReader) parseAccount(ctx core.Context, data []string) (*qifAccou
}
if line[0] == 'N' {
accountData.name = line[1:]
accountData.Name = line[1:]
} else if line[0] == 'T' {
accountData.accountType = line[1:]
accountData.AccountType = line[1:]
} else if line[0] == 'D' {
accountData.description = line[1:]
accountData.Description = line[1:]
} else if line[0] == 'L' {
accountData.creditLimit = line[1:]
accountData.CreditLimit = line[1:]
} else if line[0] == '/' {
accountData.statementBalanceDate = line[1:]
accountData.StatementBalanceDate = line[1:]
} else if line[0] == '$' {
accountData.statementBalanceAmount = line[1:]
accountData.StatementBalanceAmount = line[1:]
} else {
log.Warnf(ctx, "[qif_data_reader.parseAccount] read unsupported line \"%s\" and skip this line", line)
continue
@@ -399,27 +399,27 @@ func (r *qifDataReader) parseCategory(ctx core.Context, data []string) (*qifCate
}
if line[0] == 'N' {
categoryData.name = line[1:]
categoryData.Name = line[1:]
} else if line[0] == 'D' {
categoryData.description = line[1:]
categoryData.Description = line[1:]
} else if line[0] == 'T' {
categoryData.taxRelated = true
categoryData.TaxRelated = true
} else if line[0] == 'I' {
categoryData.categoryType = qifIncomeTransaction
categoryData.CategoryType = qifIncomeTransaction
} else if line[0] == 'E' {
categoryData.categoryType = qifExpenseTransaction
categoryData.CategoryType = qifExpenseTransaction
} else if line[0] == 'B' {
categoryData.budgetAmount = line[1:]
categoryData.BudgetAmount = line[1:]
} else if line[0] == 'R' {
categoryData.taxScheduleInformation = line[1:]
categoryData.TaxScheduleInformation = line[1:]
} else {
log.Warnf(ctx, "[qif_data_reader.parseCategory] read unsupported line \"%s\" and skip this line", line)
continue
}
}
if categoryData.categoryType == "" {
categoryData.categoryType = qifExpenseTransaction
if categoryData.CategoryType == "" {
categoryData.CategoryType = qifExpenseTransaction
}
return categoryData, nil
@@ -440,9 +440,9 @@ func (r *qifDataReader) parseClass(ctx core.Context, data []string) (*qifClassDa
}
if line[0] == 'N' {
classData.name = line[1:]
classData.Name = line[1:]
} else if line[0] == 'D' {
classData.description = line[1:]
classData.Description = line[1:]
} else {
log.Warnf(ctx, "[qif_data_reader.parseClass] read unsupported line \"%s\" and skip this line", line)
continue
+137 -137
View File
@@ -64,48 +64,48 @@ func TestQifDataReaderParse(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 2, len(actualData.bankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.bankAccountTransactions[0].date)
assert.Equal(t, "-123.45", actualData.bankAccountTransactions[0].amount)
assert.Equal(t, "2024/10/12", actualData.bankAccountTransactions[1].date)
assert.Equal(t, "+234.56", actualData.bankAccountTransactions[1].amount)
assert.Equal(t, 2, len(actualData.BankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.BankAccountTransactions[0].Date)
assert.Equal(t, "-123.45", actualData.BankAccountTransactions[0].Amount)
assert.Equal(t, "2024/10/12", actualData.BankAccountTransactions[1].Date)
assert.Equal(t, "+234.56", actualData.BankAccountTransactions[1].Amount)
assert.Equal(t, 1, len(actualData.cashAccountTransactions))
assert.Equal(t, "2024/9/1", actualData.cashAccountTransactions[0].date)
assert.Equal(t, "100.00", actualData.cashAccountTransactions[0].amount)
assert.Equal(t, "Opening Balance", actualData.cashAccountTransactions[0].payee)
assert.Equal(t, "[Wallet]", actualData.cashAccountTransactions[0].category)
assert.Equal(t, 1, len(actualData.CashAccountTransactions))
assert.Equal(t, "2024/9/1", actualData.CashAccountTransactions[0].Date)
assert.Equal(t, "100.00", actualData.CashAccountTransactions[0].Amount)
assert.Equal(t, "Opening Balance", actualData.CashAccountTransactions[0].Payee)
assert.Equal(t, "[Wallet]", actualData.CashAccountTransactions[0].Category)
assert.Equal(t, 1, len(actualData.memorizedTransactions))
assert.Equal(t, qifCheckTransactionType, actualData.memorizedTransactions[0].transactionType)
assert.Equal(t, "-123.45", actualData.memorizedTransactions[0].amount)
assert.Equal(t, "2024/10/13", actualData.memorizedTransactions[0].amortization.firstPaymentDate)
assert.Equal(t, "3", actualData.memorizedTransactions[0].amortization.totalYearsForLoan)
assert.Equal(t, "1", actualData.memorizedTransactions[0].amortization.numberOfPayments)
assert.Equal(t, "2", actualData.memorizedTransactions[0].amortization.numberOfPeriodsPerYear)
assert.Equal(t, "12.34", actualData.memorizedTransactions[0].amortization.interestRate)
assert.Equal(t, "100.45", actualData.memorizedTransactions[0].amortization.currentLoanBalance)
assert.Equal(t, "234.56", actualData.memorizedTransactions[0].amortization.originalLoanAmount)
assert.Equal(t, 1, len(actualData.MemorizedTransactions))
assert.Equal(t, qifCheckTransactionType, actualData.MemorizedTransactions[0].TransactionType)
assert.Equal(t, "-123.45", actualData.MemorizedTransactions[0].Amount)
assert.Equal(t, "2024/10/13", actualData.MemorizedTransactions[0].Amortization.FirstPaymentDate)
assert.Equal(t, "3", actualData.MemorizedTransactions[0].Amortization.TotalYearsForLoan)
assert.Equal(t, "1", actualData.MemorizedTransactions[0].Amortization.NumberOfPayments)
assert.Equal(t, "2", actualData.MemorizedTransactions[0].Amortization.NumberOfPeriodsPerYear)
assert.Equal(t, "12.34", actualData.MemorizedTransactions[0].Amortization.InterestRate)
assert.Equal(t, "100.45", actualData.MemorizedTransactions[0].Amortization.CurrentLoanBalance)
assert.Equal(t, "234.56", actualData.MemorizedTransactions[0].Amortization.OriginalLoanAmount)
assert.Equal(t, 1, len(actualData.investmentAccountTransactions))
assert.Equal(t, "2024/10/14", actualData.investmentAccountTransactions[0].date)
assert.Equal(t, "Buy", actualData.investmentAccountTransactions[0].action)
assert.Equal(t, "Test", actualData.investmentAccountTransactions[0].security)
assert.Equal(t, "12.34", actualData.investmentAccountTransactions[0].price)
assert.Equal(t, "10", actualData.investmentAccountTransactions[0].quantity)
assert.Equal(t, "-123.4", actualData.investmentAccountTransactions[0].amount)
assert.Equal(t, 1, len(actualData.InvestmentAccountTransactions))
assert.Equal(t, "2024/10/14", actualData.InvestmentAccountTransactions[0].Date)
assert.Equal(t, "Buy", actualData.InvestmentAccountTransactions[0].Action)
assert.Equal(t, "Test", actualData.InvestmentAccountTransactions[0].Security)
assert.Equal(t, "12.34", actualData.InvestmentAccountTransactions[0].Price)
assert.Equal(t, "10", actualData.InvestmentAccountTransactions[0].Quantity)
assert.Equal(t, "-123.4", actualData.InvestmentAccountTransactions[0].Amount)
assert.Equal(t, 2, len(actualData.accounts))
assert.Equal(t, "Test Account", actualData.accounts[0].name)
assert.Equal(t, "Wallet", actualData.accounts[1].name)
assert.Equal(t, 2, len(actualData.Accounts))
assert.Equal(t, "Test Account", actualData.Accounts[0].Name)
assert.Equal(t, "Wallet", actualData.Accounts[1].Name)
assert.Equal(t, 1, len(actualData.categories))
assert.Equal(t, "Test Category", actualData.categories[0].name)
assert.Equal(t, qifIncomeTransaction, actualData.categories[0].categoryType)
assert.Equal(t, 1, len(actualData.Categories))
assert.Equal(t, "Test Category", actualData.Categories[0].Name)
assert.Equal(t, qifIncomeTransaction, actualData.Categories[0].CategoryType)
assert.Equal(t, 1, len(actualData.classes))
assert.Equal(t, "Test Class", actualData.classes[0].name)
assert.Equal(t, "Foo Bar", actualData.classes[0].description)
assert.Equal(t, 1, len(actualData.Classes))
assert.Equal(t, "Test Class", actualData.Classes[0].Name)
assert.Equal(t, "Foo Bar", actualData.Classes[0].Description)
}
func TestQifDataReaderParse_AccountEntryBeforeTransaction(t *testing.T) {
@@ -137,21 +137,21 @@ func TestQifDataReaderParse_AccountEntryBeforeTransaction(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 2, len(actualData.bankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.bankAccountTransactions[0].date)
assert.Equal(t, "-123.45", actualData.bankAccountTransactions[0].amount)
assert.Equal(t, "2024/10/12", actualData.bankAccountTransactions[1].date)
assert.Equal(t, "+234.56", actualData.bankAccountTransactions[1].amount)
assert.Equal(t, 2, len(actualData.BankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.BankAccountTransactions[0].Date)
assert.Equal(t, "-123.45", actualData.BankAccountTransactions[0].Amount)
assert.Equal(t, "2024/10/12", actualData.BankAccountTransactions[1].Date)
assert.Equal(t, "+234.56", actualData.BankAccountTransactions[1].Amount)
assert.Equal(t, 1, len(actualData.cashAccountTransactions))
assert.Equal(t, "2024/9/1", actualData.cashAccountTransactions[0].date)
assert.Equal(t, "100.00", actualData.cashAccountTransactions[0].amount)
assert.Equal(t, "Opening Balance", actualData.cashAccountTransactions[0].payee)
assert.Equal(t, "[Wallet]", actualData.cashAccountTransactions[0].category)
assert.Equal(t, 1, len(actualData.CashAccountTransactions))
assert.Equal(t, "2024/9/1", actualData.CashAccountTransactions[0].Date)
assert.Equal(t, "100.00", actualData.CashAccountTransactions[0].Amount)
assert.Equal(t, "Opening Balance", actualData.CashAccountTransactions[0].Payee)
assert.Equal(t, "[Wallet]", actualData.CashAccountTransactions[0].Category)
assert.Equal(t, 2, len(actualData.accounts))
assert.Equal(t, "Test Account", actualData.accounts[0].name)
assert.Equal(t, "Wallet", actualData.accounts[1].name)
assert.Equal(t, 2, len(actualData.Accounts))
assert.Equal(t, "Test Account", actualData.Accounts[0].Name)
assert.Equal(t, "Wallet", actualData.Accounts[1].Name)
}
func TestQifDataReaderParse_EmptyContent(t *testing.T) {
@@ -188,13 +188,13 @@ func TestQifDataReaderParse_EmptyEntry(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 0, len(actualData.bankAccountTransactions))
assert.Equal(t, 0, len(actualData.cashAccountTransactions))
assert.Equal(t, 0, len(actualData.memorizedTransactions))
assert.Equal(t, 0, len(actualData.investmentAccountTransactions))
assert.Equal(t, 0, len(actualData.accounts))
assert.Equal(t, 0, len(actualData.categories))
assert.Equal(t, 0, len(actualData.classes))
assert.Equal(t, 0, len(actualData.BankAccountTransactions))
assert.Equal(t, 0, len(actualData.CashAccountTransactions))
assert.Equal(t, 0, len(actualData.MemorizedTransactions))
assert.Equal(t, 0, len(actualData.InvestmentAccountTransactions))
assert.Equal(t, 0, len(actualData.Accounts))
assert.Equal(t, 0, len(actualData.Categories))
assert.Equal(t, 0, len(actualData.Classes))
}
func TestQifDataReaderParse_UnsupportedEntryHeader(t *testing.T) {
@@ -215,9 +215,9 @@ func TestQifDataReaderParse_UnsupportedEntryHeader(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 1, len(actualData.bankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.bankAccountTransactions[0].date)
assert.Equal(t, "-123.45", actualData.bankAccountTransactions[0].amount)
assert.Equal(t, 1, len(actualData.BankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.BankAccountTransactions[0].Date)
assert.Equal(t, "-123.45", actualData.BankAccountTransactions[0].Amount)
}
func TestQifDataReaderParse_UnsupportedLine(t *testing.T) {
@@ -238,11 +238,11 @@ func TestQifDataReaderParse_UnsupportedLine(t *testing.T) {
actualData, err := reader.read(context)
assert.Nil(t, err)
assert.Equal(t, 2, len(actualData.bankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.bankAccountTransactions[0].date)
assert.Equal(t, "-123.45", actualData.bankAccountTransactions[0].amount)
assert.Equal(t, "2024/10/11", actualData.bankAccountTransactions[1].date)
assert.Equal(t, "100.00", actualData.bankAccountTransactions[1].amount)
assert.Equal(t, 2, len(actualData.BankAccountTransactions))
assert.Equal(t, "2024/10/9", actualData.BankAccountTransactions[0].Date)
assert.Equal(t, "-123.45", actualData.BankAccountTransactions[0].Amount)
assert.Equal(t, "2024/10/11", actualData.BankAccountTransactions[1].Date)
assert.Equal(t, "100.00", actualData.BankAccountTransactions[1].Amount)
}
func TestQifDataReaderParse_NewEntryHeaderAfterUnclosedEntry(t *testing.T) {
@@ -289,26 +289,26 @@ func TestQifDataReaderParseTransaction_SupportedFields(t *testing.T) {
}, false)
assert.Nil(t, err)
assert.Equal(t, "2024/10/12", actualData.date)
assert.Equal(t, "-123.45", actualData.amount)
assert.Equal(t, qifClearedStatusUnreconciled, actualData.clearedStatus)
assert.Equal(t, "100", actualData.num)
assert.Equal(t, "Foo", actualData.payee)
assert.Equal(t, "Bar", actualData.memo)
assert.Equal(t, 3, len(actualData.addresses))
assert.Equal(t, "Address 1", actualData.addresses[0])
assert.Equal(t, "Address 2", actualData.addresses[1])
assert.Equal(t, "Address 3", actualData.addresses[2])
assert.Equal(t, "Test Category", actualData.category)
assert.Equal(t, 2, len(actualData.subTransactionCategory))
assert.Equal(t, "Part1 Category", actualData.subTransactionCategory[0])
assert.Equal(t, "Part2 Category", actualData.subTransactionCategory[1])
assert.Equal(t, 2, len(actualData.subTransactionMemo))
assert.Equal(t, "Part1 Memo", actualData.subTransactionMemo[0])
assert.Equal(t, "Part2 Memo", actualData.subTransactionMemo[1])
assert.Equal(t, 2, len(actualData.subTransactionAmount))
assert.Equal(t, "-100.00", actualData.subTransactionAmount[0])
assert.Equal(t, "-23.45", actualData.subTransactionAmount[1])
assert.Equal(t, "2024/10/12", actualData.Date)
assert.Equal(t, "-123.45", actualData.Amount)
assert.Equal(t, qifClearedStatusUnreconciled, actualData.ClearedStatus)
assert.Equal(t, "100", actualData.Num)
assert.Equal(t, "Foo", actualData.Payee)
assert.Equal(t, "Bar", actualData.Memo)
assert.Equal(t, 3, len(actualData.Addresses))
assert.Equal(t, "Address 1", actualData.Addresses[0])
assert.Equal(t, "Address 2", actualData.Addresses[1])
assert.Equal(t, "Address 3", actualData.Addresses[2])
assert.Equal(t, "Test Category", actualData.Category)
assert.Equal(t, 2, len(actualData.SubTransactionCategory))
assert.Equal(t, "Part1 Category", actualData.SubTransactionCategory[0])
assert.Equal(t, "Part2 Category", actualData.SubTransactionCategory[1])
assert.Equal(t, 2, len(actualData.SubTransactionMemo))
assert.Equal(t, "Part1 Memo", actualData.SubTransactionMemo[0])
assert.Equal(t, "Part2 Memo", actualData.SubTransactionMemo[1])
assert.Equal(t, 2, len(actualData.SubTransactionAmount))
assert.Equal(t, "-100.00", actualData.SubTransactionAmount[0])
assert.Equal(t, "-23.45", actualData.SubTransactionAmount[1])
}
func TestQifDataReaderParseMemorizedTransaction_SupportedFields(t *testing.T) {
@@ -333,36 +333,36 @@ func TestQifDataReaderParseMemorizedTransaction_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, qifCheckTransactionType, actualData.transactionType)
assert.Equal(t, "2024/10/12", actualData.date)
assert.Equal(t, "-123.45", actualData.amount)
assert.Equal(t, qifClearedStatusCleared, actualData.clearedStatus)
assert.Equal(t, "100", actualData.num)
assert.Equal(t, "Foo", actualData.payee)
assert.Equal(t, "Bar", actualData.memo)
assert.Equal(t, "2024/10/13", actualData.amortization.firstPaymentDate)
assert.Equal(t, "3", actualData.amortization.totalYearsForLoan)
assert.Equal(t, "1", actualData.amortization.numberOfPayments)
assert.Equal(t, "2", actualData.amortization.numberOfPeriodsPerYear)
assert.Equal(t, "12.34", actualData.amortization.interestRate)
assert.Equal(t, "100.45", actualData.amortization.currentLoanBalance)
assert.Equal(t, "234.56", actualData.amortization.originalLoanAmount)
assert.Equal(t, qifCheckTransactionType, actualData.TransactionType)
assert.Equal(t, "2024/10/12", actualData.Date)
assert.Equal(t, "-123.45", actualData.Amount)
assert.Equal(t, qifClearedStatusCleared, actualData.ClearedStatus)
assert.Equal(t, "100", actualData.Num)
assert.Equal(t, "Foo", actualData.Payee)
assert.Equal(t, "Bar", actualData.Memo)
assert.Equal(t, "2024/10/13", actualData.Amortization.FirstPaymentDate)
assert.Equal(t, "3", actualData.Amortization.TotalYearsForLoan)
assert.Equal(t, "1", actualData.Amortization.NumberOfPayments)
assert.Equal(t, "2", actualData.Amortization.NumberOfPeriodsPerYear)
assert.Equal(t, "12.34", actualData.Amortization.InterestRate)
assert.Equal(t, "100.45", actualData.Amortization.CurrentLoanBalance)
assert.Equal(t, "234.56", actualData.Amortization.OriginalLoanAmount)
actualData, err = reader.parseMemorizedTransaction(context, []string{"KD"})
assert.Nil(t, err)
assert.Equal(t, qifDepositTransactionType, actualData.transactionType)
assert.Equal(t, qifDepositTransactionType, actualData.TransactionType)
actualData, err = reader.parseMemorizedTransaction(context, []string{"KP"})
assert.Nil(t, err)
assert.Equal(t, qifPaymentTransactionType, actualData.transactionType)
assert.Equal(t, qifPaymentTransactionType, actualData.TransactionType)
actualData, err = reader.parseMemorizedTransaction(context, []string{"KI"})
assert.Nil(t, err)
assert.Equal(t, qifInvestmentTransactionType, actualData.transactionType)
assert.Equal(t, qifInvestmentTransactionType, actualData.TransactionType)
actualData, err = reader.parseMemorizedTransaction(context, []string{"KE"})
assert.Nil(t, err)
assert.Equal(t, qifElectronicPayeeTransactionType, actualData.transactionType)
assert.Equal(t, qifElectronicPayeeTransactionType, actualData.TransactionType)
}
func TestQifDataReaderParseInvestmentTransaction_SupportedFields(t *testing.T) {
@@ -385,18 +385,18 @@ func TestQifDataReaderParseInvestmentTransaction_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "2024/10/12", actualData.date)
assert.Equal(t, "Buy", actualData.action)
assert.Equal(t, "Test", actualData.security)
assert.Equal(t, "12.34", actualData.price)
assert.Equal(t, "10", actualData.quantity)
assert.Equal(t, "-123.4", actualData.amount)
assert.Equal(t, qifClearedStatusReconciled, actualData.clearedStatus)
assert.Equal(t, "Foo", actualData.text)
assert.Equal(t, "Bar", actualData.memo)
assert.Equal(t, "Test2", actualData.commission)
assert.Equal(t, "Account Name", actualData.accountForTransfer)
assert.Equal(t, "100", actualData.amountTransferred)
assert.Equal(t, "2024/10/12", actualData.Date)
assert.Equal(t, "Buy", actualData.Action)
assert.Equal(t, "Test", actualData.Security)
assert.Equal(t, "12.34", actualData.Price)
assert.Equal(t, "10", actualData.Quantity)
assert.Equal(t, "-123.4", actualData.Amount)
assert.Equal(t, qifClearedStatusReconciled, actualData.ClearedStatus)
assert.Equal(t, "Foo", actualData.Text)
assert.Equal(t, "Bar", actualData.Memo)
assert.Equal(t, "Test2", actualData.Commission)
assert.Equal(t, "Account Name", actualData.AccountForTransfer)
assert.Equal(t, "100", actualData.AmountTransferred)
}
func TestQifDataReaderParseAccount_SupportedFields(t *testing.T) {
@@ -413,12 +413,12 @@ func TestQifDataReaderParseAccount_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "Account Name", actualData.name)
assert.Equal(t, "Account Type", actualData.accountType)
assert.Equal(t, "Some Text", actualData.description)
assert.Equal(t, "1234.56", actualData.creditLimit)
assert.Equal(t, "2024/10/12", actualData.statementBalanceDate)
assert.Equal(t, "123.45", actualData.statementBalanceAmount)
assert.Equal(t, "Account Name", actualData.Name)
assert.Equal(t, "Account Type", actualData.AccountType)
assert.Equal(t, "Some Text", actualData.Description)
assert.Equal(t, "1234.56", actualData.CreditLimit)
assert.Equal(t, "2024/10/12", actualData.StatementBalanceDate)
assert.Equal(t, "123.45", actualData.StatementBalanceAmount)
}
func TestQifDataReaderParseCategory_SupportedFields(t *testing.T) {
@@ -435,12 +435,12 @@ func TestQifDataReaderParseCategory_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "Category Name:Sub Category Name", actualData.name)
assert.Equal(t, "Some Text", actualData.description)
assert.Equal(t, true, actualData.taxRelated)
assert.Equal(t, qifIncomeTransaction, actualData.categoryType)
assert.Equal(t, "123.45", actualData.budgetAmount)
assert.Equal(t, "Test", actualData.taxScheduleInformation)
assert.Equal(t, "Category Name:Sub Category Name", actualData.Name)
assert.Equal(t, "Some Text", actualData.Description)
assert.Equal(t, true, actualData.TaxRelated)
assert.Equal(t, qifIncomeTransaction, actualData.CategoryType)
assert.Equal(t, "123.45", actualData.BudgetAmount)
assert.Equal(t, "Test", actualData.TaxScheduleInformation)
actualData2, err := reader.parseCategory(context, []string{
"NCategory Name:Sub Category Name",
@@ -449,10 +449,10 @@ func TestQifDataReaderParseCategory_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "Category Name:Sub Category Name", actualData2.name)
assert.Equal(t, "Some Text", actualData2.description)
assert.Equal(t, false, actualData2.taxRelated)
assert.Equal(t, qifExpenseTransaction, actualData2.categoryType)
assert.Equal(t, "Category Name:Sub Category Name", actualData2.Name)
assert.Equal(t, "Some Text", actualData2.Description)
assert.Equal(t, false, actualData2.TaxRelated)
assert.Equal(t, qifExpenseTransaction, actualData2.CategoryType)
actualData3, err := reader.parseCategory(context, []string{
"NCategory Name:Sub Category Name",
@@ -460,9 +460,9 @@ func TestQifDataReaderParseCategory_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "Category Name:Sub Category Name", actualData3.name)
assert.Equal(t, "Some Text", actualData3.description)
assert.Equal(t, qifExpenseTransaction, actualData3.categoryType)
assert.Equal(t, "Category Name:Sub Category Name", actualData3.Name)
assert.Equal(t, "Some Text", actualData3.Description)
assert.Equal(t, qifExpenseTransaction, actualData3.CategoryType)
}
func TestQifDataReaderParseClass_SupportedFields(t *testing.T) {
@@ -475,8 +475,8 @@ func TestQifDataReaderParseClass_SupportedFields(t *testing.T) {
})
assert.Nil(t, err)
assert.Equal(t, "Class Name", actualData.name)
assert.Equal(t, "Some Text", actualData.description)
assert.Equal(t, "Class Name", actualData.Name)
assert.Equal(t, "Some Text", actualData.Description)
}
func TestQifDataReaderParse_UnsupportedFieldsOrValues(t *testing.T) {
@@ -489,7 +489,7 @@ func TestQifDataReaderParse_UnsupportedFieldsOrValues(t *testing.T) {
"",
}, false)
assert.Nil(t, err)
assert.Equal(t, qifClearedStatusUnreconciled, actualTransactionData.clearedStatus)
assert.Equal(t, qifClearedStatusUnreconciled, actualTransactionData.ClearedStatus)
actualMemorizedTransactionData, err := reader.parseMemorizedTransaction(context, []string{
"ZTest",
@@ -497,7 +497,7 @@ func TestQifDataReaderParse_UnsupportedFieldsOrValues(t *testing.T) {
"",
})
assert.Nil(t, err)
assert.Equal(t, qifInvalidTransactionType, actualMemorizedTransactionData.transactionType)
assert.Equal(t, qifInvalidTransactionType, actualMemorizedTransactionData.TransactionType)
_, err = reader.parseInvestmentTransaction(context, []string{
"ZTest",
@@ -226,6 +226,44 @@ func TestQIFTransactionDataFileParseImportedData_ParseDayYearMonthDateFormatTime
assert.Equal(t, int64(1725494400), utils.GetUnixTimeFromTransactionTime(allNewTransactions[4].TransactionTime))
}
func TestQIFTransactionDataFileParseImportedData_ParseShortYearMonthDayDateFormatTime(t *testing.T) {
converter := QifYearMonthDayTransactionDataImporter
context := core.NewNullContext()
user := &models.User{
Uid: 1234567890,
DefaultCurrency: "CNY",
}
allNewTransactions, _, _, _, _, _, err := converter.ParseImportedData(context, user, []byte(
"!Type:Bank\n"+
"D24-09-01\n"+
"T-123.45\n"+
"^\n"+
"D24-9-2\n"+
"T-123.45\n"+
"^\n"+
"D24/9/3\n"+
"T-123.45\n"+
"^\n"+
"D24.9.4\n"+
"T-123.45\n"+
"^\n"+
"D24'9.5\n"+
"T-123.45\n"+
"^\n"), 0, nil, nil, nil, nil, nil)
assert.Nil(t, err)
assert.Equal(t, 5, len(allNewTransactions))
assert.Equal(t, int64(1725148800), utils.GetUnixTimeFromTransactionTime(allNewTransactions[0].TransactionTime))
assert.Equal(t, int64(1725235200), utils.GetUnixTimeFromTransactionTime(allNewTransactions[1].TransactionTime))
assert.Equal(t, int64(1725321600), utils.GetUnixTimeFromTransactionTime(allNewTransactions[2].TransactionTime))
assert.Equal(t, int64(1725408000), utils.GetUnixTimeFromTransactionTime(allNewTransactions[3].TransactionTime))
assert.Equal(t, int64(1725494400), utils.GetUnixTimeFromTransactionTime(allNewTransactions[4].TransactionTime))
}
func TestQIFTransactionDataFileParseImportedData_ParseInvalidTime(t *testing.T) {
converter := QifYearMonthDayTransactionDataImporter
context := core.NewNullContext()
@@ -1,7 +1,6 @@
package qif
import (
"fmt"
"strings"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
@@ -93,7 +92,7 @@ func (t *qifTransactionDataRowIterator) HasNext() bool {
return t.currentIndex+1 < len(t.dataTable.allData)
}
// Next returns the next imported data row
// Next returns the next transaction data row
func (t *qifTransactionDataRowIterator) Next(ctx core.Context, user *models.User) (daraRow datatable.TransactionDataRow, err error) {
if t.currentIndex+1 >= len(t.dataTable.allData) {
return nil, nil
@@ -119,11 +118,11 @@ func (t *qifTransactionDataRowIterator) Next(ctx core.Context, user *models.User
func (t *qifTransactionDataRowIterator) parseTransaction(ctx core.Context, user *models.User, qifTransaction *qifTransactionData) (map[datatable.TransactionDataTableColumn]string, error) {
data := make(map[datatable.TransactionDataTableColumn]string, len(qifTransactionSupportedColumns))
if qifTransaction.date == "" {
if qifTransaction.Date == "" {
return nil, errs.ErrMissingTransactionTime
}
transactionTime, err := t.parseTransactionTime(ctx, qifTransaction.date)
transactionTime, err := t.parseTransactionTime(ctx, qifTransaction.Date)
if err != nil {
return nil, err
@@ -131,37 +130,37 @@ func (t *qifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = transactionTime
if qifTransaction.amount == "" {
if qifTransaction.Amount == "" {
return nil, errs.ErrAmountInvalid
}
amount, err := utils.ParseAmount(strings.ReplaceAll(qifTransaction.amount, ",", "")) // trim thousands separator
amount, err := utils.ParseAmount(strings.ReplaceAll(qifTransaction.Amount, ",", "")) // trim thousands separator
if err != nil {
return nil, errs.ErrAmountInvalid
}
if qifTransaction.account != nil {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.account.name
if qifTransaction.Account != nil {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.Account.Name
} else {
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = ""
}
if len(qifTransaction.category) > 0 && qifTransaction.category[0] == '[' && qifTransaction.category[len(qifTransaction.category)-1] == ']' {
if qifTransaction.payee == qifOpeningBalancePayeeText { // balance modification
if len(qifTransaction.Category) > 0 && qifTransaction.Category[0] == '[' && qifTransaction.Category[len(qifTransaction.Category)-1] == ']' {
if qifTransaction.Payee == qifOpeningBalancePayeeText { // balance modification
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = qifTransactionTypeNameMapping[models.TRANSACTION_TYPE_MODIFY_BALANCE]
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.category[1 : len(qifTransaction.category)-1]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.Category[1 : len(qifTransaction.Category)-1]
} else { // transfer
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = qifTransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER]
if amount >= 0 { // transfer from [account name]
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(amount)
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.category[1 : len(qifTransaction.category)-1]
data[datatable.TRANSACTION_DATA_TABLE_ACCOUNT_NAME] = qifTransaction.Category[1 : len(qifTransaction.Category)-1]
} else { // transfer to [account name]
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-amount)
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = qifTransaction.category[1 : len(qifTransaction.category)-1]
data[datatable.TRANSACTION_DATA_TABLE_RELATED_ACCOUNT_NAME] = qifTransaction.Category[1 : len(qifTransaction.Category)-1]
}
}
} else { // income/expense
@@ -173,20 +172,20 @@ func (t *qifTransactionDataRowIterator) parseTransaction(ctx core.Context, user
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = utils.FormatAmount(-amount)
}
if strings.Index(qifTransaction.category, ":") > 0 { // category:subcategory
categories := strings.Split(qifTransaction.category, ":")
if strings.Index(qifTransaction.Category, ":") > 0 { // category:subcategory
categories := strings.Split(qifTransaction.Category, ":")
data[datatable.TRANSACTION_DATA_TABLE_CATEGORY] = categories[0]
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = categories[len(categories)-1]
} else {
data[datatable.TRANSACTION_DATA_TABLE_CATEGORY] = ""
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = qifTransaction.category
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = qifTransaction.Category
}
}
if qifTransaction.memo != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = qifTransaction.memo
} else if qifTransaction.payee != "" && qifTransaction.payee != qifOpeningBalancePayeeText {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = qifTransaction.payee
if qifTransaction.Memo != "" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = qifTransaction.Memo
} else if qifTransaction.Payee != "" && qifTransaction.Payee != qifOpeningBalancePayeeText {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = qifTransaction.Payee
}
return data, nil
@@ -223,15 +222,7 @@ func (t *qifTransactionDataRowIterator) parseTransactionTime(ctx core.Context, d
return "", errs.ErrTransactionTimeInvalid
}
if len(month) < 2 {
month = "0" + month
}
if len(day) < 2 {
day = "0" + day
}
return fmt.Sprintf("%s-%s-%s 00:00:00", year, month, day), nil
return utils.FormatYearMonthDayToLongDateTime(year, month, day)
}
func createNewQifTransactionDataTable(dateFormatType qifDateFormatType, qifData *qifData) (*qifTransactionDataTable, error) {
@@ -240,11 +231,11 @@ func createNewQifTransactionDataTable(dateFormatType qifDateFormatType, qifData
}
allData := make([]*qifTransactionData, 0)
allData = append(allData, qifData.bankAccountTransactions...)
allData = append(allData, qifData.cashAccountTransactions...)
allData = append(allData, qifData.creditCardAccountTransactions...)
allData = append(allData, qifData.assetAccountTransactions...)
allData = append(allData, qifData.liabilityAccountTransactions...)
allData = append(allData, qifData.BankAccountTransactions...)
allData = append(allData, qifData.CashAccountTransactions...)
allData = append(allData, qifData.CreditCardAccountTransactions...)
allData = append(allData, qifData.AssetAccountTransactions...)
allData = append(allData, qifData.LiabilityAccountTransactions...)
return &qifTransactionDataTable{
dateFormatType: dateFormatType,
@@ -3,6 +3,7 @@ package converters
import (
"github.com/mayswind/ezbookkeeping/pkg/converters/alipay"
"github.com/mayswind/ezbookkeeping/pkg/converters/beancount"
"github.com/mayswind/ezbookkeeping/pkg/converters/camt"
"github.com/mayswind/ezbookkeeping/pkg/converters/converter"
"github.com/mayswind/ezbookkeeping/pkg/converters/datatable"
"github.com/mayswind/ezbookkeeping/pkg/converters/default"
@@ -11,6 +12,7 @@ import (
"github.com/mayswind/ezbookkeeping/pkg/converters/fireflyIII"
"github.com/mayswind/ezbookkeeping/pkg/converters/gnucash"
"github.com/mayswind/ezbookkeeping/pkg/converters/iif"
"github.com/mayswind/ezbookkeeping/pkg/converters/mt"
"github.com/mayswind/ezbookkeeping/pkg/converters/ofx"
"github.com/mayswind/ezbookkeeping/pkg/converters/qif"
"github.com/mayswind/ezbookkeeping/pkg/converters/wechat"
@@ -47,6 +49,10 @@ func GetTransactionDataImporter(fileType string) (converter.TransactionDataImpor
return qif.QifDayMonthYearTransactionDataImporter, nil
} else if fileType == "iif" {
return iif.IifTransactionDataFileImporter, nil
} else if fileType == "camt053" {
return camt.Camt053TransactionDataImporter, nil
} else if fileType == "mt940" {
return mt.MT940TransactionDataFileImporter, nil
} else if fileType == "gnucash" {
return gnucash.GnuCashTransactionDataImporter, nil
} else if fileType == "firefly_iii_csv" {
@@ -81,6 +87,6 @@ func CreateNewDelimiterSeparatedValuesDataParser(fileType string, fileEncoding s
}
// CreateNewDelimiterSeparatedValuesDataImporter returns a new delimiter-separated values data importer according to the file type and encoding
func CreateNewDelimiterSeparatedValuesDataImporter(fileType string, fileEncoding string, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, hasHeaderLine bool, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string, geoLocationSeparator string, transactionTagSeparator string) (converter.TransactionDataImporter, error) {
return dsv.CreateNewCustomTransactionDataDsvFileImporter(fileType, fileEncoding, columnIndexMapping, transactionTypeNameMapping, hasHeaderLine, timeFormat, timezoneFormat, amountDecimalSeparator, amountDigitGroupingSymbol, geoLocationSeparator, transactionTagSeparator)
func CreateNewDelimiterSeparatedValuesDataImporter(fileType string, fileEncoding string, columnIndexMapping map[datatable.TransactionDataTableColumn]int, transactionTypeNameMapping map[string]models.TransactionType, hasHeaderLine bool, timeFormat string, timezoneFormat string, amountDecimalSeparator string, amountDigitGroupingSymbol string, geoLocationSeparator string, geoLocationOrder string, transactionTagSeparator string) (converter.TransactionDataImporter, error) {
return dsv.CreateNewCustomTransactionDataDsvFileImporter(fileType, fileEncoding, columnIndexMapping, transactionTypeNameMapping, hasHeaderLine, timeFormat, timezoneFormat, amountDecimalSeparator, amountDigitGroupingSymbol, geoLocationSeparator, geoLocationOrder, transactionTagSeparator)
}
@@ -49,13 +49,13 @@ func (c *wechatPayTransactionDataCsvFileImporter) ParseImportedData(ctx core.Con
fallback := unicode.UTF8.NewDecoder()
reader := transform.NewReader(bytes.NewReader(data), unicode.BOMOverride(fallback))
dataTable, err := c.createNewWeChatPayImportedDataTable(ctx, reader)
dataTable, err := c.createNewWeChatPayBasicDataTable(ctx, reader)
if err != nil {
return nil, nil, nil, nil, nil, nil, err
}
commonDataTable := datatable.CreateNewImportedCommonDataTable(dataTable)
commonDataTable := datatable.CreateNewCommonDataTableFromBasicDataTable(dataTable)
if !commonDataTable.HasColumn(wechatPayTransactionTimeColumnName) ||
!commonDataTable.HasColumn(wechatPayTransactionCategoryColumnName) ||
@@ -66,14 +66,14 @@ func (c *wechatPayTransactionDataCsvFileImporter) ParseImportedData(ctx core.Con
return nil, nil, nil, nil, nil, nil, errs.ErrMissingRequiredFieldInHeaderRow
}
transactionRowParser := createWeChatPayTransactionDataRowParser()
transactionDataTable := datatable.CreateNewCommonTransactionDataTable(commonDataTable, wechatPayTransactionSupportedColumns, transactionRowParser)
transactionRowParser := createWeChatPayTransactionDataRowParser(dataTable.HeaderColumnNames())
transactionDataTable := datatable.CreateNewTransactionDataTableFromCommonDataTable(commonDataTable, wechatPayTransactionSupportedColumns, transactionRowParser)
dataTableImporter := converter.CreateNewSimpleImporterWithTypeNameMapping(wechatPayTransactionTypeNameMapping)
return dataTableImporter.ParseImportedData(ctx, user, transactionDataTable, defaultTimezoneOffset, accountMap, expenseCategoryMap, incomeCategoryMap, transferCategoryMap, tagMap)
}
func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayImportedDataTable(ctx core.Context, reader io.Reader) (datatable.ImportedDataTable, error) {
func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayBasicDataTable(ctx core.Context, reader io.Reader) (datatable.BasicDataTable, error) {
csvReader := csv.NewReader(reader)
csvReader.FieldsPerRecord = -1
@@ -89,7 +89,7 @@ func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayImportedData
}
if err != nil {
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayImportedDataTable] cannot parse wechat pay csv data, because %s", err.Error())
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayBasicDataTable] cannot parse wechat pay csv data, because %s", err.Error())
return nil, errs.ErrInvalidCSVFile
}
@@ -100,7 +100,7 @@ func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayImportedData
hasFileHeader = true
continue
} else {
log.Warnf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayImportedDataTable] read unexpected line before read file header, line content is %s", strings.Join(items, ","))
log.Warnf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayBasicDataTable] read unexpected line before read file header, line content is %s", strings.Join(items, ","))
continue
}
}
@@ -126,7 +126,7 @@ func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayImportedData
}
if len(allOriginalLines) > 0 && len(items) < len(allOriginalLines[0]) {
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayImportedDataTable] cannot parse row \"index:%d\", because may missing some columns (column count %d in data row is less than header column count %d)", len(allOriginalLines), len(items), len(allOriginalLines[0]))
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayBasicDataTable] cannot parse row \"index:%d\", because may missing some columns (column count %d in data row is less than header column count %d)", len(allOriginalLines), len(items), len(allOriginalLines[0]))
return nil, errs.ErrFewerFieldsInDataRowThanInHeaderRow
}
@@ -139,11 +139,11 @@ func (c *wechatPayTransactionDataCsvFileImporter) createNewWeChatPayImportedData
}
if len(allOriginalLines) < 2 {
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayImportedDataTable] cannot parse import data, because data table row count is less 1")
log.Errorf(ctx, "[wechat_pay_transaction_data_csv_file_importer.createNewWeChatPayBasicDataTable] cannot parse import data, because data table row count is less 1")
return nil, errs.ErrNotFoundTransactionDataInFile
}
dataTable := csvdatatable.CreateNewCustomCsvImportedDataTable(allOriginalLines)
dataTable := csvdatatable.CreateNewCustomCsvBasicDataTable(allOriginalLines)
return dataTable, nil
}
@@ -31,10 +31,11 @@ const wechatPayTransactionDataStatusRefundName = "退款"
// weChatPayTransactionDataRowParser defines the structure of wechat pay transaction data row parser
type weChatPayTransactionDataRowParser struct {
existedOriginalDataColumns map[string]bool
}
// Parse returns the converted transaction data row
func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataTable *datatable.CommonTransactionDataTable, dataRow datatable.CommonDataRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
func (p *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models.User, dataRow datatable.CommonDataTableRow, rowId string) (rowData map[datatable.TransactionDataTableColumn]string, rowDataValid bool, err error) {
if dataRow.GetData(wechatPayTransactionTypeColumnName) != wechatPayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] &&
dataRow.GetData(wechatPayTransactionTypeColumnName) != wechatPayTransactionTypeNameMapping[models.TRANSACTION_TYPE_EXPENSE] &&
dataRow.GetData(wechatPayTransactionTypeColumnName) != wechatPayTransactionTypeNameMapping[models.TRANSACTION_TYPE_TRANSFER] {
@@ -44,15 +45,15 @@ func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models
data := make(map[datatable.TransactionDataTableColumn]string, len(wechatPayTransactionSupportedColumns))
if dataTable.HasOriginalColumn(wechatPayTransactionTimeColumnName) {
if p.hasOriginalColumn(wechatPayTransactionTimeColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TIME] = dataRow.GetData(wechatPayTransactionTimeColumnName)
}
if dataTable.HasOriginalColumn(wechatPayTransactionCategoryColumnName) {
if p.hasOriginalColumn(wechatPayTransactionCategoryColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_SUB_CATEGORY] = dataRow.GetData(wechatPayTransactionCategoryColumnName)
}
if dataTable.HasOriginalColumn(wechatPayTransactionAmountColumnName) {
if p.hasOriginalColumn(wechatPayTransactionAmountColumnName) {
amount, success := utils.ParseFirstConsecutiveNumber(dataRow.GetData(wechatPayTransactionAmountColumnName))
if !success {
@@ -63,9 +64,9 @@ func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models
data[datatable.TRANSACTION_DATA_TABLE_AMOUNT] = amount
}
if dataTable.HasOriginalColumn(wechatPayTransactionDescriptionColumnName) && dataRow.GetData(wechatPayTransactionDescriptionColumnName) != "" && dataRow.GetData(wechatPayTransactionDescriptionColumnName) != "/" {
if p.hasOriginalColumn(wechatPayTransactionDescriptionColumnName) && dataRow.GetData(wechatPayTransactionDescriptionColumnName) != "" && dataRow.GetData(wechatPayTransactionDescriptionColumnName) != "/" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(wechatPayTransactionDescriptionColumnName)
} else if dataTable.HasOriginalColumn(wechatPayTransactionProductNameColumnName) && dataRow.GetData(wechatPayTransactionProductNameColumnName) != "" && dataRow.GetData(wechatPayTransactionProductNameColumnName) != "/" {
} else if p.hasOriginalColumn(wechatPayTransactionProductNameColumnName) && dataRow.GetData(wechatPayTransactionProductNameColumnName) != "" && dataRow.GetData(wechatPayTransactionProductNameColumnName) != "/" {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = dataRow.GetData(wechatPayTransactionProductNameColumnName)
} else {
data[datatable.TRANSACTION_DATA_TABLE_DESCRIPTION] = ""
@@ -73,13 +74,13 @@ func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models
relatedAccountName := ""
if dataTable.HasOriginalColumn(wechatPayTransactionRelatedAccountColumnName) {
if p.hasOriginalColumn(wechatPayTransactionRelatedAccountColumnName) {
relatedAccountName = dataRow.GetData(wechatPayTransactionRelatedAccountColumnName)
}
statusName := ""
if dataTable.HasOriginalColumn(wechatPayTransactionStatusColumnName) {
if p.hasOriginalColumn(wechatPayTransactionStatusColumnName) {
statusName = dataRow.GetData(wechatPayTransactionStatusColumnName)
}
@@ -91,7 +92,7 @@ func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models
localeTextItems := locales.GetLocaleTextItems(locale)
if dataTable.HasOriginalColumn(wechatPayTransactionTypeColumnName) {
if p.hasOriginalColumn(wechatPayTransactionTypeColumnName) {
data[datatable.TRANSACTION_DATA_TABLE_TRANSACTION_TYPE] = dataRow.GetData(wechatPayTransactionTypeColumnName)
if dataRow.GetData(wechatPayTransactionTypeColumnName) == wechatPayTransactionTypeNameMapping[models.TRANSACTION_TYPE_INCOME] {
@@ -132,7 +133,20 @@ func (t *weChatPayTransactionDataRowParser) Parse(ctx core.Context, user *models
return data, true, nil
}
// createWeChatPayTransactionDataRowParser returns wechat pay transaction data row parser
func createWeChatPayTransactionDataRowParser() datatable.CommonTransactionDataRowParser {
return &weChatPayTransactionDataRowParser{}
func (p *weChatPayTransactionDataRowParser) hasOriginalColumn(columnName string) bool {
_, exists := p.existedOriginalDataColumns[columnName]
return exists
}
// createWeChatPayTransactionDataRowParser returns wechat pay transaction data row parser
func createWeChatPayTransactionDataRowParser(headerColumnNames []string) datatable.CommonTransactionDataRowParser {
existedOriginalDataColumns := make(map[string]bool, len(headerColumnNames))
for i := 0; i < len(headerColumnNames); i++ {
existedOriginalDataColumns[headerColumnNames[i]] = true
}
return &weChatPayTransactionDataRowParser{
existedOriginalDataColumns: existedOriginalDataColumns,
}
}
+111
View File
@@ -0,0 +1,111 @@
package core
import (
"fmt"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
// FiscalYearStart represents the fiscal year start date as a uint16 (month: high byte, day: low byte)
type FiscalYearStart uint16
// Fiscal Year Start Date Type
const (
FISCAL_YEAR_START_DEFAULT FiscalYearStart = 0x0101 // January 1
FISCAL_YEAR_START_MIN FiscalYearStart = 0x0101 // January 1
FISCAL_YEAR_START_MAX FiscalYearStart = 0x0C1F // December 31
FISCAL_YEAR_START_INVALID FiscalYearStart = 0xFFFF // Invalid
)
var MONTH_MAX_DAYS = []uint8{
uint8(31), // January
uint8(28), // February (Disallow fiscal year start on leap day)
uint8(31), // March
uint8(30), // April
uint8(31), // May
uint8(30), // June
uint8(31), // July
uint8(31), // August
uint8(30), // September
uint8(31), // October
uint8(30), // November
uint8(31), // December
}
// NewFiscalYearStart creates a new FiscalYearStart from month and day values
func NewFiscalYearStart(month uint8, day uint8) (FiscalYearStart, error) {
if !isValidFiscalYearMonthDay(month, day) {
return 0, errs.ErrFormatInvalid
}
return FiscalYearStart(uint16(month)<<8 | uint16(day)), nil
}
// GetMonthDay extracts the month and day from FiscalYearType
func (f FiscalYearStart) GetMonthDay() (uint8, uint8, error) {
if f < FISCAL_YEAR_START_MIN || f > FISCAL_YEAR_START_MAX {
return 0, 0, errs.ErrFormatInvalid
}
// Extract month and day (month in high byte, day in low byte)
month := uint8(f >> 8)
day := uint8(f & 0xFF)
if !isValidFiscalYearMonthDay(month, day) {
return 0, 0, errs.ErrFormatInvalid
}
return month, day, nil
}
// String returns a string representation of FiscalYearStart in MM/DD format
func (f FiscalYearStart) String() string {
month, day, err := f.GetMonthDay()
if err != nil {
return "Invalid"
}
return fmt.Sprintf("%02d-%02d", month, day)
}
// isValidFiscalYearMonthDay returns whether the specified month and day is valid
func isValidFiscalYearMonthDay(month uint8, day uint8) bool {
return uint8(1) <= month && month <= uint8(12) && uint8(1) <= day && day <= MONTH_MAX_DAYS[int(month)-1]
}
// FiscalYearFormat represents the fiscal year format as a uint8
type FiscalYearFormat uint8
// Fiscal Year Format Type Name
const (
FISCAL_YEAR_FORMAT_DEFAULT FiscalYearFormat = 0
FISCAL_YEAR_FORMAT_STARTYYYY_ENDYYYY FiscalYearFormat = 1
FISCAL_YEAR_FORMAT_STARTYYYY_ENDYY FiscalYearFormat = 2
FISCAL_YEAR_FORMAT_STARTYY_ENDYY FiscalYearFormat = 3
FISCAL_YEAR_FORMAT_ENDYYYY FiscalYearFormat = 4
FISCAL_YEAR_FORMAT_ENDYY FiscalYearFormat = 5
FISCAL_YEAR_FORMAT_INVALID FiscalYearFormat = 255 // Invalid
)
// String returns a textual representation of the long date format enum
func (f FiscalYearFormat) String() string {
switch f {
case FISCAL_YEAR_FORMAT_DEFAULT:
return "Default"
case FISCAL_YEAR_FORMAT_STARTYYYY_ENDYYYY:
return "StartYYYY-EndYYYY"
case FISCAL_YEAR_FORMAT_STARTYYYY_ENDYY:
return "StartYYYY-EndYY"
case FISCAL_YEAR_FORMAT_STARTYY_ENDYY:
return "StartYY-EndYY"
case FISCAL_YEAR_FORMAT_ENDYYYY:
return "EndYYYY"
case FISCAL_YEAR_FORMAT_ENDYY:
return "EndYY"
case FISCAL_YEAR_FORMAT_INVALID:
return "Invalid"
default:
return fmt.Sprintf("Invalid(%d)", int(f))
}
}
+128
View File
@@ -0,0 +1,128 @@
package core
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
func TestNewFiscalYearStart_ValidMonthDay(t *testing.T) {
testCases := []struct {
month uint8
day uint8
expected FiscalYearStart
}{
{1, 1, 0x0101}, // January 1
{4, 15, 0x040F}, // April 15
{7, 1, 0x0701}, // July 1
{12, 31, 0x0C1F}, // December 31
}
for _, tc := range testCases {
fiscal, err := NewFiscalYearStart(tc.month, tc.day)
assert.Nil(t, err)
assert.Equal(t, tc.expected, fiscal)
}
}
func TestNewFiscalYearStart_InvalidMonthDay(t *testing.T) {
testCases := []struct {
month uint8
day uint8
}{
{0, 1}, // Month 0 (invalid)
{13, 1}, // Month 13 (invalid)
{1, 0}, // Day 0 (invalid)
{1, 32}, // Day 32 (invalid for January)
{2, 30}, // Day 30 (invalid for February)
{2, 29}, // Day 29 (leap day not permitted)
{4, 31}, // Day 31 (invalid for April)
{6, 31}, // Day 31 (invalid for June)
{9, 31}, // Day 31 (invalid for September)
{11, 32}, // Day 32 (invalid for November)
{255, 15}, // Invalid month
{5, 255}, // Invalid day
}
for _, tc := range testCases {
fiscal, err := NewFiscalYearStart(tc.month, tc.day)
assert.Equal(t, FiscalYearStart(0), fiscal)
assert.Equal(t, errs.ErrFormatInvalid, err)
}
}
func TestGetMonthDay_ValidFiscalYearStart(t *testing.T) {
testCases := []struct {
fiscalYear FiscalYearStart
month uint8
day uint8
}{
{0x0101, 1, 1}, // January 1st
{0x0C1F, 12, 31}, // December 31st
{0x0701, 7, 1}, // July 1st
{0x040F, 4, 15}, // April 15th
}
for _, tc := range testCases {
month, day, err := tc.fiscalYear.GetMonthDay()
assert.Nil(t, err)
assert.Equal(t, tc.month, month)
assert.Equal(t, tc.day, day)
}
}
func TestGetMonthDay_InvalidFiscalYearStart(t *testing.T) {
testCases := []struct {
fiscalYear FiscalYearStart
}{
{0x0000}, // 0/0 (invalid)
{0x0D01}, // Month 13 (invalid)
{0x0100}, // Day 0 (invalid)
{0x0120}, // January 32 (invalid)
{0x021D}, // February 29 (not permitted)
{0x021E}, // February 30 (invalid)
{0x041F}, // April 31 (invalid)
{0x061F}, // June 31 (invalid)
{0x091F}, // September 31 (invalid)
{0x0B20}, // November 32 (invalid)
{0xFF01}, // Invalid month
{0x01FF}, // Invalid day
{0}, // Zero value
}
for _, tc := range testCases {
month, day, err := tc.fiscalYear.GetMonthDay()
assert.Equal(t, uint8(0), month)
assert.Equal(t, uint8(0), day)
assert.Equal(t, errs.ErrFormatInvalid, err)
}
}
func TestFiscalYearStart_String(t *testing.T) {
testCases := []struct {
fiscalYear FiscalYearStart
expected string
}{
{0x0101, "01-01"}, // January 1st
{0x0C1F, "12-31"}, // December 31st
{0x0701, "07-01"}, // July 1st
{0x040F, "04-15"}, // April 15th
{0x021D, "Invalid"}, // February 29th (leap day not permitted)
{0x0000, "Invalid"}, // Invalid date
{0x0D01, "Invalid"}, // Invalid month
{0x0120, "Invalid"}, // Invalid day
}
for _, tc := range testCases {
assert.Equal(t, tc.expected, tc.fiscalYear.String())
}
}
func TestFiscalYearStartConstants(t *testing.T) {
assert.Equal(t, FiscalYearStart(0xFFFF), FISCAL_YEAR_START_INVALID)
assert.Equal(t, FiscalYearStart(0x0101), FISCAL_YEAR_START_DEFAULT)
assert.Equal(t, FiscalYearStart(0x0101), FISCAL_YEAR_START_MIN)
assert.Equal(t, FiscalYearStart(0x0C1F), FISCAL_YEAR_START_MAX)
}
+3
View File
@@ -15,6 +15,9 @@ type MiddlewareHandlerFunc func(*WebContext)
// ApiHandlerFunc represents the api handler function
type ApiHandlerFunc func(*WebContext) (any, *errs.Error)
// JSONRPCApiHandlerFunc represents the api handler function
type JSONRPCApiHandlerFunc func(*WebContext, *JSONRPCRequest) (any, *errs.Error)
// EventStreamApiHandlerFunc represents the event stream api handler function
type EventStreamApiHandlerFunc func(*WebContext) *errs.Error
+177
View File
@@ -0,0 +1,177 @@
package core
import (
"regexp"
"strconv"
"strings"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
// IPPattern represents a pattern for matching IP addresses, either IPv4 or IPv6
type IPPattern struct {
Pattern string
regex *regexp.Regexp
}
// Match returns if the given IP address matches the pattern
func (p *IPPattern) Match(ip string) bool {
if p.regex == nil {
return false
}
return p.regex.MatchString(ip)
}
// GobEncode returns the encoded data for this IP pattern
func (p *IPPattern) GobEncode() ([]byte, error) {
return []byte(p.Pattern), nil
}
// GobDecode decodes the data into the IP pattern
func (p *IPPattern) GobDecode(data []byte) error {
pattern := string(data)
if pattern == "" {
p.Pattern = ""
p.regex = nil
return nil
}
newPattern, err := ParseIPPattern(pattern)
if err != nil {
return err
}
p.Pattern = newPattern.Pattern
p.regex = newPattern.regex
return nil
}
// ParseIPPattern parses the given IP address pattern and returns an IPPattern object
func ParseIPPattern(ipPattern string) (*IPPattern, error) {
if ipPattern == "" {
return nil, nil
}
hasDot := false
hasSemicolon := false
for i := 0; i < len(ipPattern); i++ {
ch := rune(ipPattern[i])
if ch == '.' { // may be IPv4
if hasSemicolon {
return nil, errs.ErrInvalidIpAddressPattern
}
hasDot = true
} else if ch == ':' { // may be IPv6
if hasDot {
return nil, errs.ErrInvalidIpAddressPattern
}
hasSemicolon = true
}
}
if hasDot {
return ParseIPv4Pattern(ipPattern)
} else if hasSemicolon {
return ParseIPv6Pattern(ipPattern)
} else {
return nil, errs.ErrInvalidIpAddressPattern
}
}
// ParseIPv4Pattern parses the given IPv4 address pattern and returns an IPPattern object
func ParseIPv4Pattern(ipPattern string) (*IPPattern, error) {
items := strings.Split(ipPattern, ".")
if len(items) != 4 {
return nil, errs.ErrInvalidIpAddressPattern
}
regexBuilder := strings.Builder{}
regexBuilder.WriteRune('^')
for i := 0; i < len(items); i++ {
item := strings.TrimSpace(items[i])
if item == "*" {
regexBuilder.WriteString("[0-9]{1,3}")
} else if item == "" {
return nil, errs.ErrInvalidIpAddressPattern
} else {
num, err := strconv.Atoi(item)
if err != nil || num < 0 || num > 255 {
return nil, errs.ErrInvalidIpAddressPattern
}
regexBuilder.WriteString(item)
}
if i < len(items)-1 {
regexBuilder.WriteRune('\\')
regexBuilder.WriteRune('.')
}
}
regexBuilder.WriteRune('$')
regex, err := regexp.Compile(regexBuilder.String())
if err != nil {
return nil, errs.ErrInvalidIpAddressPattern
}
return &IPPattern{
Pattern: ipPattern,
regex: regex,
}, nil
}
// ParseIPv6Pattern parses the given IPv6 address pattern and returns an IPPattern object
func ParseIPv6Pattern(ipPattern string) (*IPPattern, error) {
items := strings.Split(ipPattern, ":")
if len(items) < 2 || len(items) > 8 {
return nil, errs.ErrInvalidIpAddressPattern
}
regexBuilder := strings.Builder{}
regexBuilder.WriteRune('^')
for i := 0; i < len(items); i++ {
item := strings.TrimSpace(items[i])
if item == "*" {
regexBuilder.WriteString("[0-9a-fA-F]{1,4}")
} else if i < len(items)-1 && item == "" {
// Do Nothing
} else {
num, err := strconv.ParseInt(item, 16, 32)
if err != nil || num < 0 || num > 0xFFFF {
return nil, errs.ErrInvalidIpAddressPattern
}
regexBuilder.WriteString(item)
}
if i < len(items)-1 {
regexBuilder.WriteRune(':')
}
}
regexBuilder.WriteRune('$')
regex, err := regexp.Compile(regexBuilder.String())
if err != nil {
return nil, errs.ErrInvalidIpAddressPattern
}
return &IPPattern{
Pattern: ipPattern,
regex: regex,
}, nil
}
+135
View File
@@ -0,0 +1,135 @@
package core
import (
"bytes"
"encoding/gob"
"testing"
"github.com/stretchr/testify/assert"
"github.com/mayswind/ezbookkeeping/pkg/errs"
)
func TestIPPattern_GobEncode(t *testing.T) {
pattern, err := ParseIPPattern("192.168.1.*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
var buf bytes.Buffer
err = gob.NewEncoder(&buf).Encode(pattern)
assert.Nil(t, err)
newPattern := &IPPattern{}
err = gob.NewDecoder(bytes.NewBuffer(buf.Bytes())).Decode(newPattern)
assert.Nil(t, err)
assert.NotNil(t, newPattern)
assert.Equal(t, pattern.Pattern, newPattern.Pattern)
assert.Equal(t, pattern.regex.String(), newPattern.regex.String())
assert.True(t, newPattern.Match("192.168.1.1"))
assert.True(t, newPattern.Match("192.168.1.255"))
}
func TestParseIPPattern(t *testing.T) {
pattern, err := ParseIPPattern("")
assert.Nil(t, err)
assert.Nil(t, pattern)
pattern, err = ParseIPPattern("invalid")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPPattern("192.1:2:3.4")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPPattern("0:0:0:0:0:0:1.2.3.4") // not support IPv6 with embedded IPv4
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPPattern("192.168.1.*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("192.168.1.1"))
assert.True(t, pattern.Match("192.168.1.255"))
assert.False(t, pattern.Match("192.168.2.1"))
pattern, err = ParseIPPattern("2001:db8::*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("2001:db8::1"))
assert.True(t, pattern.Match("2001:db8::ffff"))
assert.False(t, pattern.Match("2001:db9::1"))
}
func TestParseIPv4Pattern(t *testing.T) {
pattern, err := ParseIPv4Pattern("192.168.1.1")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("192.168.1.1"))
assert.False(t, pattern.Match("192.168.1.2"))
pattern, err = ParseIPv4Pattern("192.168.*.1")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("192.168.1.1"))
assert.True(t, pattern.Match("192.168.255.1"))
assert.False(t, pattern.Match("192.168.1.2"))
pattern, err = ParseIPv4Pattern("*.*.*.*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("0.0.0.0"))
assert.True(t, pattern.Match("255.255.255.255"))
pattern, err = ParseIPv4Pattern("256.256.256.256")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPv4Pattern("1.2.3")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPv4Pattern("1.2.3.4.5")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPv4Pattern("a.b.c.d")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
}
func TestParseIPv6Pattern(t *testing.T) {
pattern, err := ParseIPv6Pattern("2001:db8:85a3:8d3:1319:8a2e:370:7348")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("2001:db8:85a3:8d3:1319:8a2e:370:7348"))
assert.False(t, pattern.Match("2001:db8:85a3:8d3:1319:8a2e:370:7349"))
pattern, err = ParseIPv6Pattern("2001:db8::*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("2001:db8::0"))
assert.True(t, pattern.Match("2001:db8::ffff"))
assert.False(t, pattern.Match("2001:db9::0"))
pattern, err = ParseIPv6Pattern("::*")
assert.Nil(t, err)
assert.NotNil(t, pattern)
assert.True(t, pattern.Match("::1"))
assert.True(t, pattern.Match("::2"))
assert.False(t, pattern.Match(":1:1"))
pattern, err = ParseIPv6Pattern("2001:db8:85a3:8d3:1319:8a2e:370:7348:extra")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPv6Pattern("g001:db8:85a3:8d3")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
pattern, err = ParseIPv6Pattern("2001:db8:")
assert.Equal(t, errs.ErrInvalidIpAddressPattern, err)
assert.Nil(t, pattern)
}
+95
View File
@@ -0,0 +1,95 @@
package core
import "encoding/json"
// JSONRPCVersion defines the version of JSON-RPC protocol
const JSONRPCVersion = "2.0"
// JSONRPCRequest represents the JSON-RPC 2.0 request
type JSONRPCRequest struct {
JSONRPC string `json:"jsonrpc"`
Method string `json:"method"`
Params json.RawMessage `json:"params,omitempty"`
ID any `json:"id,omitempty"`
}
// JSONRPCResponse represents the JSON-RPC 2.0 response
type JSONRPCResponse struct {
JSONRPC string `json:"jsonrpc"`
Result any `json:"result,omitempty"`
Error *JSONRPCError `json:"error,omitempty"`
ID any `json:"id,omitempty"`
}
// JSONRPCError represents the JSON-RPC 2.0 error object
type JSONRPCError struct {
Code int `json:"code"`
Message string `json:"message"`
Data any `json:"data,omitempty"`
}
// JSONRPCParseError represents the "Parse error" in JSON-RPC 2.0
var JSONRPCParseError = &JSONRPCError{
Code: -32700,
Message: "Parse error",
Data: nil,
}
// JSONRPCMethodNotFoundError represents the "Method not found" error in JSON-RPC 2.0
var JSONRPCMethodNotFoundError = &JSONRPCError{
Code: -32601,
Message: "Method not found",
Data: nil,
}
// JSONRPCInvalidParamsError represents the "Invalid params" error in JSON-RPC 2.0
var JSONRPCInvalidParamsError = &JSONRPCError{
Code: -32602,
Message: "Invalid params",
Data: nil,
}
// JSONRPCInternalError represents the "Internal error" in JSON-RPC 2.0
var JSONRPCInternalError = &JSONRPCError{
Code: -32603,
Message: "Internal error",
Data: nil,
}
// NewJSONRPCResponse creates a new JSON-RPC response with the result
func NewJSONRPCResponse(id any, result any) *JSONRPCResponse {
return &JSONRPCResponse{
JSONRPC: JSONRPCVersion,
Result: result,
Error: nil,
ID: id,
}
}
// NewJSONRPCErrorResponse creates a new JSON-RPC error response
func NewJSONRPCErrorResponse(id any, err *JSONRPCError) *JSONRPCResponse {
return &JSONRPCResponse{
JSONRPC: JSONRPCVersion,
Result: nil,
Error: &JSONRPCError{
Code: err.Code,
Message: err.Message,
Data: nil,
},
ID: id,
}
}
// NewJSONRPCErrorResponseWithCause creates a new JSON-RPC error response
func NewJSONRPCErrorResponseWithCause(id any, err *JSONRPCError, cause string) *JSONRPCResponse {
return &JSONRPCResponse{
JSONRPC: JSONRPCVersion,
Result: nil,
Error: &JSONRPCError{
Code: err.Code,
Message: err.Message,
Data: cause,
},
ID: id,
}
}

Some files were not shown because too many files have changed in this diff Show More