\P 11 shows 9 decimal places? - Kdb+ / qStudio

Home Forums Kdb+ / qStudio \P 11 shows 9 decimal places?

This topic contains 1 reply, has 2 voices, and was last updated by  matt 3 years, 3 months ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #111272

    Jim
    Member

    Why when I set \P to 11 does it show less decimal places than that?

    `
    q)\P 11
    q)a:101.1234567890123
    q)a
    101.12345679
    `

    #111274

    matt
    Member

    Hi Jim!

    I’m by no means an expert in kdb+, but I believe it’s because \P sets the number of digits to display, not the number of decimal places. You might want to use Q.fmt and .Q.f instead if you want to follow a more specific format for output (note: these return strings). See http://code.kx.com/wiki/Reference/DisplayPrecision for more info.

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.