decimal problem

Hi all

I a very strange effect when using different decimal settings
and calculating some values:

DECLARE @Decimal1 decimal(5, 2)
DECLARE @Decimal2 decimal(30, 2)

SET @Decimal1 = 0.2
SET @Decimal2 = 0.2

SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal1))
SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal2))

The first result is correct (0.00000000025832000),
the second is wrong, why ???

Thanks for any comments

Best regards
Frank Uray
0
Utf
4/28/2010 1:16:13 PM
sqlserver.programming 1873 articles. 0 followers. Follow

1 Replies
804 Views

Similar Articles

[PageSpeed] 49

Hi Frank

This is what happens when you multiple these precision&scales

Precision
15 + 30 = 45 + 1 = 46
Scale
2+15= 17

so (46,17)

Please see below articles:

http://msdn.microsoft.com/en-us/library/ms190476.aspx?ppud=4
http://blogs.msdn.com/sqlprogrammability/archive/2006/03/29/564110.aspx

* The result precision and scale have an absolute maximum of 38. When a 
result precision is greater than 38, the corresponding scale is reduced to 
prevent the integral part of a result from being truncated.

Jon

"Frank Uray" wrote:

> Hi all
> 
> I a very strange effect when using different decimal settings
> and calculating some values:
> 
> DECLARE @Decimal1 decimal(5, 2)
> DECLARE @Decimal2 decimal(30, 2)
> 
> SET @Decimal1 = 0.2
> SET @Decimal2 = 0.2
> 
> SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal1))
> SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal2))
> 
> The first result is correct (0.00000000025832000),
> the second is wrong, why ???
> 
> Thanks for any comments
> 
> Best regards
> Frank Uray
0
Utf
4/28/2010 1:32:05 PM
Reply:

Similar Artilces:

SOP NonInventory Unit Cost 6 Decimal Places
Is there any simple way to increase the currency decimal places to greater than the current highest value of 5? Thanx Scott, I don't believe there is any easy or even moderately difficult way to do this, as the field types used to store amounts in GP only go up to 5 decimal places. -- Victoria Yudin Dynamics GP MVP Flexible Solutions - home of GP Reports http://www.flex-solutions.com/gpreports.html blog: http://www.victoriayudin.com "Scott Rutledge" <sersoar@hotmail.com> wrote in message news:E102B81B-8555-400E-843C-240C58794294@microsoft.com... > Is there any...

Another Standby & System Stanby Problem
Up until about a week ago my wife was able to put her desktop PC into Standby Mode (Start/Turn Off Computer/Standby), and the "Power Schemes" worked so that the computer would go into "System Standby" after 1 hour of non use. Now only the screen saver comes on correctly but the computer will not go into System Standby after 1 hour of non use nor can the computer be put into Standby manually (Start/Turn Off Computer/Standby). When she tried to put the computer into Standby mode manually, it turns off the monitor, starts to turn off the computer but it comes r...

Excel Search Problem
I have five columns (A,B,C,D,E) of which I have a varying number of row per time. Column A will contain numbers from 1+ and Column E wil contain numbers and text strings. What I want to do is have a colum E that has a formula that for each row use its corrisponding value in and search All values in column E for the same value and if that A contains a value listed in column E somewhere return the label "True and if not found return "False". Example A B C D E F 1 - - - 2000 FALSE 2 - - - 2001 FALSE 2000 - - - STORM TRUE 200...

Need solution to formula problem
Help please! I have two rows of numbers with some cell that will contain 'N/A". If the top row is highest I need it highlighted yellow if it is not highest or is "N/A" the cell needs to remain with no fill. I can do a condtional formatting that works great until a cell contains the "N/A". Is there a formula that I can do in conditional formatting or some other options that will solve this problem. As suggested in your multi-post in .programming (please do not multi-post) Try this .. Select rows 1 & 2, and apply cond format Cond1's formula: =A$1=&q...

Fetch XML problem
Hi, all. I have a problem that is caused by a limitation of the fetch XML schema: I want to calculate the prices for a list products according to a special pricelist by the default unit of measure. I found no way to select unique product and priceleveldetail details for the products because I can only give one join attribute (productid or uomid) for the link-entity. Example: <fetch mapping='logical'> <entity name='product'> <attribute name='productid'/> <attribute name='name'/> <attribute name='price'/>...

Reduce currency decimals
Hi, Is there any tool which will reduce the currency decimals on our Great Plains system. We want to reduce our system currency decimals. Thank you. KT I'm not sure if this is what you mean, but there's Tools >> Utilities >> Inventory >> Change Decimal Places This utility can change the decimal places for items. I don't know of any system-wide change for decimal places. Eleni "KT" wrote: > Hi, > > Is there any tool which will reduce the currency decimals on our Great > Plains system. We want to reduce our system currency decimals....

Outlook 2000 "Out of memory or system resources" problems
OS: Win XP Pro CPU/Ram: 1.3M Presario Laptop 256MB RAM I run Win XP Pro on Pentium 1.3M with 256MB Ram, I'm getting "out of memory or low in system resources" error while using OUTLOOK 2000. Some folders don't open and and I can't move items between folders. This started just 1 week ago. I've tried updating Office to with SR1 update. It was successful though I got one error "cannot apply patch to C:\config.msi\pt472.tmp....." saying patch may have been applied by other setup... I've tried reinstalling my McAfee virusscan, repairing office(whic...

Connectivity Problems
Sorry about the length of this post, but I have no idea what I'm talking about, let alone explain it! We have two sites (Head Office and Remote) with a Netgear FVS338 VPN firewall at each end. The remote site connects via a Terminal Server and runs a couple of EPOS systems. Our setup in the Head Office is thus: Workstation -> SBS 2003 R2 Standard -> FVS338 VPN Firewall -> Internet Router Recently, the VPN keeps dropping and if I set up a permanent ping I can see that several times a day we get 7 or 8 pings timing out. This is enough to kill the EPOS connect...

Font dialog Sample Window problem
Am using a CFontDialog derived class (CInwinFontDlg) to allow user to select new font/size etc. But for some reason cannot get the text in Sample window to either show or change when font attribute selected. The following is how my font dlg is constructed. CInwinFontDlg dlg(&lf, CF_SCREENFONTS| CF_INITTOLOGFONTSTRUCT| CF_FIXEDPITCHONLY| CF_SHOWHELP | CF_ENABLETEMPLATE); Does one have to control the look and appearance or does the Common Windows' font dialog do this with the which CF flag? -- Kevin Kohler INWIN Development Manager Response Technology, Inc. Kevin, How you constru...

sd reader problem with optima notebook
i am using a 32 bit version of vista ultimate. the inbuilt sd reader is completley unresponsive. I do also have a problem with no drivers installed for PCI Flash Memory . I am wondering how do i get my hands on some drivers to fix this problem? I dont have a manufacturer drivers disk and the site was of no help. L40 Optima Centoris V662. "Insane" <Insane@discussions.microsoft.com> wrote in message news:F04AC29F-ECD6-410A-BBD4-DA401B06DD3E@microsoft.com... >i am using a 32 bit version of vista ultimate. the inbuilt sd reader is > completley unresponsive. Your questi...

SBS Site Problems w/ CRM
After upgrading from 1.2 to 3.0, we have major problems with our SBS websites. 1) CRM is setup to use the IP address of the server and this site runs fine. 2) Upgrading knocked out our "companyweb" site installed by SBS ("service unavailable") 3) SQL Reporting doesn't work because it's trying to use the default address which is already used by CRM, RWW, companyweb, etc. I'm really confused about how to sort out all the websites. How can all these sites run on one IP address? So much for MS making this install nice and trouble free for SBS users! -- Bra...

decimal problem
Hi all I a very strange effect when using different decimal settings and calculating some values: DECLARE @Decimal1 decimal(5, 2) DECLARE @Decimal2 decimal(30, 2) SET @Decimal1 = 0.2 SET @Decimal2 = 0.2 SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal1)) SELECT (CONVERT(decimal(15, 15), 0.000000000322900) * (1 - @Decimal2)) The first result is correct (0.00000000025832000), the second is wrong, why ??? Thanks for any comments Best regards Frank Uray Hi Frank This is what happens when you multiple these precision&scales Precision 15 + 3...

table problems
when i try to open an attachment...my publisher freezes up with a mesage that reads... the table is too large for the workspace because of a printer or style change, your table has become larger than publisher's workspace. to solve this problem, you can delete your table, or remove some text from your table. note: publisher may have locked the table size as a resuilt of this action. if your table does not expand to accommodate the text, click grow to fit text on the table menu. What are they talking about??? do i need to take the excel spreadsheet out?? or is it soemthing els...

Problem with ADO
I am using a Microsoft Jet Engine database. While testing an ADO implementation, I found that I ran into trouble if I repeatedly tried to update the field with the 'too large' data; eventually ADO became convinced that I had never issued an Edit or AddNew. Of course, Edit does not exist in ADO; the edit is implied when you try to set a field. To make sure the error wasn't elsewhere, I have removed all wrapper functions and created this test case that reproduces the error using primitives. I have discovered that issuing a CancelUpdate() clears the error, but then any other edits...

Problem with paths with spaces SUDO
I have this function... function sud([scriptblock]$SomethingToExecuteInElevation,[switch]$NoExit){ $location ="`"`"`""+ (Get-Location) +"`"`"`""; if(!$NoExit){ $command = "-command &{set-Location $location; & $SomethingToExecuteInElevation;}" Start-Process powershell -Verb Runas -ArgumentList $command } else{ $command = "-NoExit -command &{set-Location $location; & $SomethingToExecuteInElevation;}" Start-Process powershell -Verb Runas -ArgumentList $command }} the problem i...

registry problem
I was trying to create a registry key for my dialog project and I added SetRegistryKey(_T("Simple Alarm Clock")); in BOOL CAlarm_ClockApp::InitInstance() , but nothing gets written to the registry. Is there anything else I need to do in order to get my dialog project to write to the registry? Z.K. That simply sets the registry key for the application. You still have to write something there. You can then use WriteProfileInt/WriteProfileString to write something at that key. -- Ajay Kalra [MVP - VC++] ajaykalra@yahoo.com "Z.K." <nospam@nospam.net> wro...

problem in mail
We are facing some problem in moving the users from one domain to other .For example the below user is moved from FMGNT domain to VPNT domain . When we moved his mailbox It was showing correctly mail box is moved to VPNT domain But once his account has been moved when we checked his mailbox his mail box is shown in FMG NT any solution Please It was showing correctly like the below screen But once his account has been moved when we checked his mailbox is at FMGNT please find the below screen shot Make sure that the new domain has ha...

Decimal
In a laboratory test we do the temperature has to be reported to the nearest 0.5�C. Can anyone let me know how to format it so that when the result is a whole number, it leaves the decimal point out? eg 22�C or 22.5�C (not 22.�C or 22.0�C) Alan "Alan Cocks" <alandrob@netspace.net.au> wrote in message news:btlqhd$19fi$1@otis.netspace.net.au... > In a laboratory test we do the temperature has to be reported to the > nearest 0.5�C. > Can anyone let me know how to format it so that when the result is a whole > number, it leaves the decimal point out? &g...

Synchronisation problem
I have a HP iPAQ 2200 PDA and I synchronise it with Outlook. If I create an appointment in my iPAQ it synchronises correctly and the appointment appears in Outlook. However, if I create an appointment in Outlook even after synchronisation the appointment will not appear in my iPAQ. Has anyone else experienced this problem? Any ideas how to fix it? I appreciate and help or advice given. Thanks in advance! Probably best to post this in m.p.pocketpc or ..pocketpc.activesync as it isn't really an Outlook issue.... Neil Hindry wrote: > I have a HP iPAQ 2200 PDA and I synchronise ...

Problems with upgrading
I am running 10.4.6 on a Mac G5 dual 2 gig. Upon launcing a MS Office 2004 application today I took notice of the available upgrade to 11.2.6 and proceded with it. This is where things get wacky. I did the upgrade and promptly encountered an error message stating I was missing a file and installation could not continue. Great. Turns out that all of Office apps were wiped out due to this non completed upgrade. Not a biggy. I uninstalled MS OFFICE and rebuilt permissions. Reinstalled MS OFFICE 2004 and tried to get my user data (email and contacts) back into Entourage. No such luck. Er...

password problem
I had saved a doc in excel and password protected it. I have forgotten the password now and do not know how to open it. Any suggestions?? Hi search Google for 'Excel password recovery'. You'll get many hits :-) -- Regards Frank Kabel Frankfurt, Germany sam wrote: > I had saved a doc in excel and password protected it. I have > forgotten the password now and do not know how to open it. Any > suggestions?? ...

Problem with Outlook 2007 client
Hello All, I have an user that gets the following message opening his Outlook 2007 sp1 (and latest patches): "An error occured loading Microsoft CRM functionality. Try restarting Microsoft Outlook. Contact you system administrator if error persists". And the error persists. And I am the admin, and having no clue what could it be. :o) Tried googling on it, and trying different things (from changing his account in CRM, to uninstalling Office and CRM client on his machine) - no luck. He can open the CRM using web - no problems. In the Application Event Log: --------------...

simple namespace deserialize problem
Hello I want to deserialize the following xml <anyRoot xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://ogsa.globus.org/any-service"><QName>abc</QName></anyRoot using TextReader tr = new StreamReader("c:\\test.xml") anyRootType rootType = (anyRootType)serializer.Deserialize(tr) tr.Close() but always get the exception Message: There is an error in XML document (1, 2) Message: <anyRoot xmlns='http://ogsa.globus.org/any-service'> was not expected If I r...

VBA code and decimals
I have some VBA code as this: rrows=lastrow/4 if lastrow is 138 then rrows should be 34.5 but I always get 34(no decimal) How do I fix that Thanks I just did this and got 34.5 Sub lastrowdecimal() lastrow = 138 rrows = lastrow / 4 MsgBox rrows End Sub -- Don Guillett SalesAid Software dguillett1@austin.rr.com "pcor" <pcor@discussions.microsoft.com> wrote in message news:F21838BC-42AA-48A3-9B0C-D2139287A187@microsoft.com... >I have some VBA code as this: > rrows=lastrow/4 > > if lastrow is 138 then rrows should be 34.5 but I always get 34(no > decimal) >...

Email forwrading problem
I am running Ex2k3 on Win2k3. My problem is that users who send email from a hotmail account, have their mail delivered to an external internet account with no problem. But users who send email from a yahoo account get immediate delivery failure message. Is there any list I need to add yahoo or any feature I need to turn on or off? By the way I have turned on Reverse DNS. I would really appreciate any help on this. On 25 May 2006 03:12:33 -0700, "john" <johnbahran@hotmail.com> wrote: >I am running Ex2k3 on Win2k3. My problem is that users who send email >from a hotmail...