Welcome to XDQSO’s documentation!

The XDQSO code accompanies the XDQSO/XDQSOz papers for quasar classification and photometric redshift estimation. It allows you to calculate photometric quasar probabilities to mimick SDSS-III’s BOSS quasar target selection or to calculate photometric redshifts for quasars, using any combination of SDSS optical, GALEX, ultraviolet, UKIDSS near-IR, and WISE mid-IR photometry

Contents:

Introduction

To download the code use either

wget -qO- https://github.com/xdqso/xdqso/archive/v0.6.tar.gz | tar xvz

or

git clone https://github.com/xdqso/xdqso.git

Installation only requires you to set the environment variable XDQSODATA to the data directory of the distribution. EvilUPS setup is available.

Code is available either as xdqso or as xdqsoz. For most purposes you will want to use the xdqsoz functions: these allow you to calculate photometric redshifts and quasar probabilities for arbitrary redshift ranges. The xdqsoz routines are the functions used to create the photometric quasar catalog. If you want to mimick SDSS-III’s BOSS quasar target selection or easily calculate probabilites in broad redshift bins you want to use the xdqso functions instead.

The functions xdqso_calculate_prob and xdqsoz_calculate_prob calculate photometric quasar probabilities. The former can only do this in three redshift ranges (z \(<\) 2.2; 2.2 \(\leq\) z \(\leq\) 3.5; z \(>\) 3.5), while the latter accomodates arbitrary redshift ranges.

Photometric redshifts can be calculated using the xdqsoz_zpdf and xdqsoz_eval_zpdf functions. The former prepares the parameters of the one-dimensional redshift PDF for individual objects, the latter then allows you to evaluate this PDF. The function xdqsoz_calculate_prob_andz wraps this functionality into the quasar probability calculation, including the redshift PDF sampled in increments of 0.01 in the output structure.

One can also use xdqsoz_marginalize_colorzprob to integrate the redshift PDF over arbitrary redshift ranges:

out= xdqsoz_marginalize_colorzprob(zmin,zmax,flux,flux_ivar,norm=totlike)

Input is dereddened psfflux and psfflux_ivar (to deredden you can use the functions xdqso_sdss_deredden and xdqso_sdss_deredden_error) and a min and max redshift; output is the marginalized likelihood (marginalized over redshift). Setting norm=totlike returns the total quasar likelihood. If you then multiply the ‘pqso’ from the photometric quasar catalog below by the ratio of out and totlike you get the desired redshift-range probability (since the prior and the denominator do not change). When calculating quasar probabilities in many bins this is much faster than calling xdqsoz_calculate_prob repeatedly because you do not recalculate star likelihoods and priors each time.

XDQSOz photometric quasar catalog

The original version of the SDSS-XDQSO DR8 photometric quasar catalog that does not include WISE information is available at

http://cosmo.nyu.edu/~jb2777/qsocat/xdqsoz_pqso0.5_imag21.5-nobadu.fits.gz

http://cosmo.nyu.edu/~jb2777/qsocat/README_pqso0.5_imag21.5-nobadu

This catalog is based on the same principle as the XDQSO method for BOSS quasar selection, but uses a slightly different algorithm (XDQSOz) for calculating quasar probabilities that also permits us to obtain photometric redshifts; it also allows quasar probabilities to be calculated quickly for arbitrary redshift ranges (see the accompanying code below).

The original catalog is a simple cut on P(quasar) > 0.5 for all objects that pass the BOSS quasar selection flag cuts, limited further to i_0 < 21.5 mag and with some bad u-columns in the SDSS imaging data masked. We have performed some first tests of the clustering of the objects in the catalog, which shows that the level of stellar contamination is small (< 10%), but we have yet to break this up by redshift range, etc., and perform further tests. So exercise caution when using the catalog (especially at low Galactic latitude, since the SEGUE stripes are included), and please let us know if you find any problems.

An updated version of the catalog is available at

http://www.mpia.de/homes/joe/xdqsozcat_galex_ukidss_wise_p20.fits.gz

http://www.mpia.de/homes/joe/README_pqso0.2_wise

which includes updated probabilities incorporporating WISE fluxes, and photometric redshift PDFs for all objects with P(quasar) > 0.2. Like the first catalog, it only includes objects that pass the BOSS quasar selection flag cuts and objects with i_0 < 21.5. There are tags to indicate if an object falls within the SDSS bright star mask, a region of bad SDSS photometry, an area with bad u-columns, or near contaminated WISE data. The same precautions as above apply to the new catalog. A catalog containing quasar/star probabilities for all point sources in SDSS DR8 is available upon request.

IDL code

Contents:

xdqso_calculate_prob (in,/dereddened,/galex,/ukidss,/wise)

calculate the extreme-deconvolution XDQSO QSO probability

Input:

in - structure containing PSFFLUX, PSFFLUX_IVAR, EXTINCTION

Keywords:

dereddened - psfflux, and psfflux_ivar is already dereddened

galex - GALEX fluxes are included in input structure, with tags NUV, FUV, NUV_ivar, and FUV_ivar. GALEX fluxes are in nanomaggies

ukidss - UKIDSS fluxes are included in input structure, with tags APERCSIFLUX3_Y, APERCSIFLUX3_J, APERCSIFLUX3_H, APERCSIFLUX3_K, APERCSIFLUX3ERR_Y, APERCSIFLUX3ERR_J, APERCSIFLUX3ERR_H, APERCSIFLUX3ERR_K. Fluxes/errors are in SI units.

wise - WISE fluxes are included in input structure, with tags w1_nanomaggies, w2_nanomaggies, w1_nanomaggies_ivar, w2_nanomaggies_ivar. Fluxes are in Vega nanomaggies.

Output:

structure containing pqso, … (see XDQSO catalog description)

History:

010-04-30 - Written - Bovy (NYU)

2014-04-02 - Added WISE, GALEX, UKIDSS - DiPompeo (UWyo)

xdqsoz_calculate_prob (in,zmin,zmax,/dereddened,/galex,/ukidss,wise)

calculate the extreme-deconvolution probability ratio, marginalizing over an arbitrary redshift range

Input:

in - structure containing PSFFLUX, PSFFLUX_IVAR, EXTINCTION

zmin, zmax - lower, upper bound of redshift interval

Keywords:

dereddened - psfflux, and psfflux_ivar are already dereddened

galex - GALEX fluxes are included in input structure, with tags NUV, FUV, NUV_ivar, and FUV_ivar. GALEX fluxes are in nanomaggies

ukidss - UKIDSS fluxes are included in input structure, with tags APERCSIFLUX3_Y, APERCSIFLUX3_J, APERCSIFLUX3_H, APERCSIFLUX3_K, APERCSIFLUX3ERR_Y, APERCSIFLUX3ERR_J, APERCSIFLUX3ERR_H, APERCSIFLUX3ERR_K. Fluxes/errors are in SI units.

wise - WISE fluxes are included in input structure, with tags w1_nanomaggies, w2_nanomaggies, w1_nanomaggies_ivar, w2_nanomaggies_ivar. Fluxes are in Vega nanomaggies.

Output:

out - structure containing pqso, …

History:

2010-04-30 - Written - Bovy (NYU)

2010-05-29 - Added Galex - Bovy

2010-10-30 - Added UKIDSS - Bovy

2014-03-31 - Added WISE - DiPompeo (UWyo)

xdqsoz_eval_zpdf (z,zmean,zcovar,zamp)

evaluate the photometric redshift PDF for a given redshift given means, covars, and amps

Input:

z - redshift [nz]

zmean, zcovar, zamp - from xdqsoz_zpdf

Output:

p(z)

History:

2011-01-18 - Written - Bovy (NYU)

xdqsoz_marginalize_colorzprob (zmin,zmax,flux,flux_ivar,/galex,/ukidss,/wise,norm=norm,/log)

marginalize the probability of a relative flux + redshift (not a color) over redshift

Input:

zmin, zmax - redshift

flux - [nfluxes] or [nfluxes,ndata] array of fluxes

flux_ivar - [nfluxes] or [nfluxes,ndata] array of flux_ivars

Keywords:

galex - use GALEX fits

ukidss - use UKIDSS fits

wise - use WISE fits

log - calculate log

Output:

number or array of probabilities

Optional Output:

norm - normalization factor (likelihood marginalized over redshift 0 to infinity)

History:

2011-01-16 - Written - Bovy (NYU)

2014-03-31 - Added WISE - DiPompeo (UWyo)

xdqsoz_peaks (flux,flux_ivar,nzs=nzs,peak_threshold=peak_threshold,/galex,/ukidss,/wise,/plot,peakz=peakz,xdqsoz=xdqsoz)

calculate the number of peaks of a zpdf as well as the MAP z

Input:

flux - dereddened flux

flux_ivar - dereddened flux_ivar

Optional Input:

nzs - number of points to sample the PDF at

peak_threshold - threshold for defining a peak (contiguous region with p above peak_threshold)

Keywords:

galex - use GALEX fits

ukidss - use UKIDSS fits

wise - use WISE fits

plot - make QS plot

Output:

number of peaks

Optional Output:

peakz - MAP z

xdqsoz - structure containing {peakz,peakprob,peakfwhm,otherz,otherprob,otherfwhm} for all peaks

History:

2011-01-18 - Written - Bovy (NYU)

2014-03-31 - Added WISE - DiPompeo (UWyo)

xdqsoz_qso_track (z,i=i,/galex,/ukidss,/wise)

calculate the mean quasar locus

Input:

z - redshift or array of redshifts [N]

Optional Input:

i= dereddened i-band magnitude

Keywords:

galex - use GALEX fits

ukidss - use UKIDSS fits

wise - use WISE fits

Output:

mags[ndim,N] - array of apparent magnitudes (ugriz[NUV,FUV,YJHK])

History:

2011-04-01 - Written - Bovy (NYU)

2014-04-02 - Added WISE - DiPompeo (UWyo)

xdqsoz_zpdf, flux, flux_ivar, /galex, /ukidss, /wise, zmean=zmean, zcovar=zcovar, zamp=zamp

calculate the photometric redshift pdf using XDQSOz

Input:

flux - [nfluxes] or [nfluxes,ndata] array of fluxes

flux_ivar - [nfluxes] or [nfluxes,ndata] array of flux_ivars

Keywords:

galex - use GALEX fits

ukidss - use UKIDSS fits

wise - use WISE fits

Output:

zmean - [ngauss,ndata] array of means

zcovar - [ngauss,ndata] array of covars

zamp - [ngauss,ndata] array of amplitudes

History:

2011-01-18 - Written - Bovy (NYU)

2014-04-02 - Added WISE - DiPompeo (UWyo)

xdqsoz_calculate_prob_andz (in,zmin,zmax,/dereddened,/galex,/ukidss,/wise)

The same as xdqsoz_calculate_prob, with xdqsoz_zpdf wrapped in to simultaneously calculate z PDF

Input:

in - structure containing PSFFLUX, PSFFLUX_IVAR, EXTINCTION

zmin, zmax - lower, upper bound of redshift interval

Keywords:

dereddened - psfflux, and psfflux_ivar are already dereddened

galex - GALEX fluxes are included in psfflux, psfflux_ivar, and extinction; use them

ukidss - use UKIDSS (like /galex)

wise - use WISE (like /galex)

Output:

out - structure containing pqso, … , z array from zmin to zmax in 0.01 increments, z PDF at each value of z.

History:

2014-03-31 - Written - DiPompeo (UWyo)

Acknowledging XDQSO

Please cite the relevant papers among the following:

BOSS CORE target selection paper (also cite Ross et al. 2011): Think outside the color box: probabilistic target selection and the SDSS-XDQSO quasar targeting catalog, Bovy, J., et al., 2010, ApJ, 729, 141 [ApJ] [ADS]

Photometric redshifts: Photometric redshifts and quasar probabilities from a single, data-driven generative model, Bovy, J., et al., 2011, ApJ, 749, 41 [ApJ] [ADS]

Updated XDQSO with WISE: Quasar Probabilities and Redshifts from WISE mid-IR through GALEX UV Photometry, DiPompeo,M.A., et al., 2015, MNRAS, 452, 3124 [MNRAS] [ADS]

Catalog paper: The SDSS-XDQSO photometric quasar catalog, Myers, A. D., et al., 2015, in preparation

XD methodology paper: Extreme deconvolution: inferring complete distribution functions from noisy, heterogeneous and incomplete observations, Bovy, J., Hogg, D. W., & Roweis, S. T., 2011, AOAS, 5, 2B, 1657 [AOAS] [ADS]