fix double.roundToPrecision#35
Open
UliPlabst wants to merge 2 commits into
Open
Conversation
|
Have added task with this PR to our typeutils tasks backlog. It would be reviewed soon |
|
Also @UliPlabst please update your changes, now they are out-of-date |
Author
|
@helen508light I updated the branch. Can you merge it? |
|
@UliPlabst |
Author
|
okay thanks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes #34. The issue turned out to be not the rounding itself but the calling of
detectPrecisionin a wrong way. Let's look at the implementation ofdetectPrecision:powerbi-visuals-utils-typeutils/src/double.ts
Lines 472 to 487 in c6485ad
As I understand if it get's passed a precision that is not null it just returns it. If precision is null then it detects the precision based on the boundary values x and y. If the precision is null it returns DEFAULT_PRECISION.
So the call
powerbi-visuals-utils-typeutils/src/double.ts
Line 356 in c6485ad
in double.roundToPrecision does not make sense. As I understand it should pass x instead of DEFAULT_PRECISION.
I also added some more test cases to verify my problems were fixed. All tests pass for me.