Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Rounding issue in QTP11
#1
Solved: 10 Years, 8 Months, 4 Weeks ago
I have run into a rounding issue and I'm hoping someone can clear this one up for me.
I have several calculations happening inside my QTP script. The first is rounding to 2 decimal places and getting the number Fee = 68.75. This is good. Then I derive another number off this by...

Fee = Payment * (.044)
Fee = Round(Fee, 2)

The purpose of this is for a checkpoint to validate a calculated number in my application. The result of this calculation is 3.025, and rounded this should be 3.03. 3.03 is what my application comes up with, but QTP is coming up with 3.02.

Help!
Reply
#2
Solved: 10 Years, 8 Months, 4 Weeks ago
Code:
Fee = FormatNumber(Fee, 2)
Reply
#3
Solved: 10 Years, 8 Months, 4 Weeks ago
ok. so that seemed to work, sort of. That fixed the rounding issue I was having but created another problem.
Someday I'd like to understand why Round means something different to QTP than it does the rest of the world.
So here is my problem now.
When I use 'FormatNumber' and then try to add those numbers together I get a string, rather than a new number.
ex.
Payment = 68.75
Payment = FormatNumber(Payment, 2)
Fee = Payment * (.044)
Fee = FormatNumber(Fee, 2)
Total = Fee + Payment
But the result is...
68.753.03
and it should be 68.75+3.03 which is 71.78

I'm obviously missing something pretty obvious.

Reply
#4
Solved: 10 Years, 8 Months, 4 Weeks ago
Code:
Total =cdbl(Fee)+cdbl(Payment)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Issues in QTP11.0 with .Net Add-in Nagaraju6 1 2,688 06-29-2016, 02:13 PM
Last Post: Ankesh
  Unable to generate Scripts using QTP11 Divya Roopa 1 1,971 03-05-2014, 09:07 AM
Last Post: basanth27
  Browser Compatability testing using QTP11 naveen51 1 1,919 12-16-2013, 10:33 AM
Last Post: supputuri
Sad Empty Server description SAP GUI 7.20 with QTP11 sylvester618 2 4,551 09-10-2013, 06:05 PM
Last Post: Ankur
  Patches List of QTP11 vinod123 1 2,254 07-15-2013, 08:21 PM
Last Post: Ankur

Forum Jump:


Users browsing this thread: 1 Guest(s)