+- Micro Focus QTP (UFT) Forums (https://www.learnqtp.com/forums)
+-- Forum: Micro Focus UFT (earlier known as QTP) (https://www.learnqtp.com/forums/Forum-Micro-Focus-UFT-earlier-known-as-QTP)
+--- Forum: UFT / QTP Beginners (https://www.learnqtp.com/forums/Forum-UFT-QTP-Beginners)
+--- Thread: Rounding issue in QTP11 (/Thread-Rounding-issue-in-QTP11)
Rounding issue in QTP11 - Brian.Osborne - 07-20-2011
I have run into a rounding issue and I'm hoping someone can clear this one up for me.
I have several calculations happening inside my QTP script. The first is rounding to 2 decimal places and getting the number Fee = 68.75. This is good. Then I derive another number off this by...
Fee = Payment * (.044)
Fee = Round(Fee, 2)
The purpose of this is for a checkpoint to validate a calculated number in my application. The result of this calculation is 3.025, and rounded this should be 3.03. 3.03 is what my application comes up with, but QTP is coming up with 3.02.
Help!
RE: Rounding issue in QTP11 - rajpes - 07-20-2011
Code:
Fee = FormatNumber(Fee, 2)RE: Rounding issue in QTP11 - Brian.Osborne - 07-21-2011
ok. so that seemed to work, sort of. That fixed the rounding issue I was having but created another problem.
Someday I'd like to understand why Round means something different to QTP than it does the rest of the world.
So here is my problem now.
When I use 'FormatNumber' and then try to add those numbers together I get a string, rather than a new number.
ex.
Payment = 68.75
Payment = FormatNumber(Payment, 2)
Fee = Payment * (.044)
Fee = FormatNumber(Fee, 2)
Total = Fee + Payment
But the result is...
68.753.03
and it should be 68.75+3.03 which is 71.78
I'm obviously missing something pretty obvious.
RE: Rounding issue in QTP11 - rajpes - 07-21-2011
Code:
Total =cdbl(Fee)+cdbl(Payment)