Korean VLBI Network (KVN)

Status of Daejeon Correlator

1. Daejeon Correlator

Daejeon Correlator
Figure 1. Block diagram of Daejeon Correlator.

Figure 1 shows the conceptual block diagram of Daejeon Correlator. There are several VLBI data playbacks which will be used in our combined VLBI network, such as Mark5B, VERA2000, OCTADISK and optical fiber which shall be connected in near future. Some of them have the VSI-H compatible interface, but the others take the different interface for the data transmission. And they have also their own maximum data recording/playing back rates respectively. To absorb all of these differences and in homogeneity among these existing VLBI data playback systems, the Raw VLBI Data Buffer (RVDB) was introduced, which is a big data server with many large RAID disks and several kind of VLBI data interfaces. The VLBI Correlation Subsystem (VCS) shall receive the VLBI data from the RVDB system, shall calculate the correlation between the every possible pairs of data inputs with proper control parameters given from the monitor and control operation computer, and then shall dump the correlation results into the data archive system. Data archive system, is also a kind of data server, which is used to capture the correlated data output from the VCS and to save them as a structured file system. Finally there is also the correlator control and operation software for overall system.

1.1 Playback System

KVN is now using the Mark5B system for recording and playback the observed data. KASI participated in Mark5B development with Haystack Observatory as a member of international consortium. It can support the VSI compatible and RAID-based HDD storage system. Recording and playing back speed is 1024 Mbps. DIR2000 is widely used in VERA for recording and playback with 1024 Mbps. Recently the manufacturer of DIR2000 had been stopped to manufacture, so NAOJ developed new playing back system named VERA2000, which is modified by DIR1000H system for only supporting playing back. For improving observation and correlation efficiency, VERA uses OCTADISK system for KaVA observations from 2014. So Daejeon Correlator also adopted OCTADISK system, which has the same functions of OCTADDB installed in RVDB (Refer 1.2).

1.2 RVDB system

RVDB system, which is developed by NAOJ, consists of Data Input Output (DIO) (currently named OCTAVIA), 10 GbE switch, and Disk Data Buffer (DDB) (currently namedOCTADDB). It is able to record the data with maximum 2048 Mbps, and can playback the observed data to the correlator with nominal 2048 Mbps. So, it has afunction such as 2048 Mbps recorder and playback at the same time. As shown inFigure 1, the different types of playback systems are used in Daejeon Correlator. So, thepurpose of the RVDB system is that it is able to adjust the data format andeasily synchronize the data during playback and maintain the buffering betweenrecorder speed and correlation speed.

1.3 VLBI Correlation Subsystem

The main specification of VCS is described in Table 1. The VCS has a capability toprocess the total 120 cross-correlation and 16 auto-correlation intended formaximum 16 stations, maximum 8192 Mbps (4-streams x 1 Gsps/2-bit/64 MHz clock) input data rates per station. The design architecture for correlatoris FX-based and it will use the variable length of FFT (Fast Fourier Transform)to maintain the 0.05 km/sec resolution of velocity at 22 GHz. The maximum delayis ±36,000 km and maximum fringe tracking is 1.075 kHz. The number offrequency channel per correlation output is 8,192.

Table 1. Specification of Daejeon Correlator.
# of antennas 16
# of inputs/antenna 4 bands(4Fx1P, 2Fx2P, 1Fx2P+2Fx1P)
Max. # of corr./input 120 cross + 16 auto
Sub-array 2 case(12+4, 8+8)
Bandwidth 512 MHz
Sampling, Digitization 1 Gsps by 2bit/sample
Maximum data rate/antenna 2 Gbps VSI-H (32parallels, 64 MHz clock)
Maximum delay compensation ±36,000km
Maximum fringe tracking 1.075 kHz
Design architecture FX-type with FPGA
FFT word length 16+16 bits fixed point for real, imaginary
Integration time 25.6 msec ~ 10.24 sec
Data output channels 8192 channels
Data output rate Maximum 1.4GB/sec at 25.6msec integration time

1.4 Correlated VLBI Data Buffer (CVDB) (old name : Data Archive System)

The basic architecture is a CPU cluster connected with infiniband. For KaVA, the first phase data archive system with about 119TB capacity has been implemented.It has four 10 Gbit Ethernet input port to connect withVCS output and one 10 GbE Ethernet port is connectedwith data file system for sharing the disk with each other. We have a plan toincrease the system capacity for support the EAVN in near future. CODA filesystem is used in data archive system for making the file system from correlatedraw data, which is revised with ccCODA 2.0 library used in Mitaka FX correlator with some modification. New CODA filessystem based on the ccCODA 2.1 library was installed.Figure 2 shows the view of Daejeon Correlator installed at Korea-Japan CorrelationCenter (KJCC) located in KASI, Daejeon, Korea.

Daejeon Correlator installed at Korea-Japan Correlation Center
Figure 2. Daejeon Correlator installed at Korea-Japan Correlation Center.

2. Correlation Processing

2.1 Correlation mode and Integration time

Daejeon Correlator is currently able to support the following correlation modes.

Table 2. Correlation Modes in Daejeon Correlator
Obs. Mode Total Data Rate Bandwidth/sub-band # of sub-bands Minimum Accum. Time # of Freq. Channels/sub-band
C5 1024 Mbps 16 MHz 16 1.6384 sec 8192
C4 1024 Mbps 32 MHz 8 0.8192 sec 8192
C3 1024 Mbps 64 MHz 4 0.4096 sec 8192
C2 1024 Mbps 128 MHz 2 0.2048 sec 8192
C1 1024 Mbps 256 MHz 1 0.1024 sec 8192
W1 2048 Mbps 512 MHz 1 0.0512 sec 8192
W2 4096 Mbps 512 MHz 2 0.0512 sec 8192
W4 8192 Mbps 512 MHz 4 0.0512 sec 8192

2.2 CODA/FITS

Daejeon Correlator will basically support following frequency channel for preparing FITS file.

  • Basic output channel of correlator : 8192 frequency channel
  • Continuum : 128 frequency channel(64 channel integration)
  • Spectral line : 512 frequency channel(16 channel integration)

2.3 FITS delivery

KJCC will deliver FITS file to PI by using FTP server or mobile disk.

2.4 Archiving policy

KJCC will organize the archiving policy for observation data, CODA and FITS file as below.

  • Observation data : basically 2 months, and then it will be released.
  • CODA : if correlated data will be used for astrometry or geodesy, it will be permanently stored at CODA server. Otherwise, the correlated raw data and CODA file system will be deleted after receiving the response from PI.
  • FITS : it will be permanently archived at Archiving server.

3. Performance of Daejeon Correlator (Continuum/Spectral-line)

3.1 Data Analysis (@AIPS)

In order to verify the performance of Daejeon correlator, data analysis was conducted.For the comparison Daejeon Correlator, Mitaka FX and DiFX correlator were used for data correlation. Observation experiments were listed in Table 3. We confirmed that results of Daejeon Correlator hadconsistent and correct value. Here we will summarize the results for R11027Band K11017A. Procedure of data analysis was performed as below. And the same parameter wasused for all correlation result during analysis. Analysis for spectral-line wasadopted with following procedure, but analysis for continuum was excluded for 9and 10 stages.

Table 3. Observation experiments list.
Experiment Object Source Recorder Comparison correlator
R11027B Daejeon Correlator evaluation Continuum, Spectral line DIR2K, Mark5B MTK FX, DiFX
R11026A Long time phase monitoring Continuum DIR1K, DIR2K, Mark5B MTK FX
K11017A 2 frequency simultaneous observation Continuum Mark5B DiFX
K12098C 4 frequency simultaneous observation Continuum Mark5B DiFX

3.2 Analysis result for continuum (R11027B)

3.2.1 Spectrum of raw data

Figure 3 shows the spectrum shape after completion of FITLD,MSORT, USUBA and INDXR task based on FITS file generated by Daejeon Correlator and DiFX. Source is 3C454.3 for continuum with Yonsei and Ulsan baseline. Phase for all of 16 IF are continuously changed. The phase slope for DiFX islower than Daejeon Correlator, but we think this is caused by adopting the clock-offsetprecisely while DiFX correlation. After fringe fitting, these values will be deleted and there is no serious problem incurrent phase slope.

Spectrum shape of 16IF continuum for Yonsei-Ulsan baseline
Figure 3. Spectrum shape of 16IF continuum for Yonsei-Ulsan baseline.

3.2.2 Gain amplitude

We looked into the variation of GAIN for each source for wholeobservation time after ACCOR. And then in figure 4, gain amplitude wascontinuously stable for source with long time integration. But in case ofsource with short integration time, the gain amplitude pattern for Daejeon Correlator wasunstable for time range(between 22:00 and 03:00 for SgrA, Sgr2M) compared with DiFX.This phenomenon was appeared in only KVN station, this is caused with datasynchronization while playing back. This phenomenon only occurred beginning ofscan about 2~4sec, and the other time range is stable. We recommended thecontinuous recording of Mark5B while observation.

Variation style of gain amplitude for all stations with 9th IF. Left(Daejeon Correlator)  Variation style of gain amplitude for all stations with 9th IF. Center(MTK FX), Right(DiFX)
Figure 4. Variation style of gain amplitude for all stations with 9th IF. Left(Daejeon Correlator), Center(MTK FX), Right(DiFX).

3.2.3 SNR, Delay, Rate after fringe fitting

Figure 5 shows the SNR, Delay and Rate for each baseline,respectively. The reference was set as Ulsan station. Integration time forfringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strongradio source in continuum sources and phase pattern is clearly remarkable. Forcomparison with DiFX, all of the patterns for eachresult are almost similar without any problems.

 SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator)   SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator)   SNR, Delay, Rate after FRING for 1st IF of Yonsei. Bottom(DiFX)
Figure 5. SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator), Bottom(DiFX).

3.2.4 Closure phase

By using previous fringe fitting result, Closure phase for3C454.3 were obtained and showed in figure 6. We confirmed that closure phasewas existed within about 5 degree by calculating with variation of phase forwhole observation time and by integrating 50~64 channels value of 1st IF forboth Daejeon Correlator and DiFX.

Closure phase after fringe fitting. Upper(Daejeon Correlator)  Closure phase after fringe fitting. Bottom(DiFX)
Figure 6. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).

3.2.5 Spectrum shape after fringe fitting

Figure 7 shows the spectrum shape after fringe fitting. We draw again the spectrum in order to check phase residual by adopting the valueof delay, rate after fringe fitting. For more detail comparison, the range ofphase was enlarged to 10 degree and only 8 IFs were shown. Phase residual forspectrum of Daejeon Correlator and DiFX was almost same with 0 value.

Spectrum shape after fringe fitting. Left(Daejeon Correlator)  Spectrum shape after fringe fitting. Right(DiFX)
Figure 7. Spectrum shape after fringe fitting. Left(Daejeon Correlator), Right(DiFX).

3.2.6 Imaging performance

New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. The 2-dimensional image map was generated using new result file andis shown in figure 8. In figure 8, only KVN 3 stations were used for comparisonto DiFX. Flux for Daejeon Correlator is 0.75 Jy/beam,which is less than about 12% compared with 0.84 Jy/beamof DiFX, but the dynamic range is almost same value.

2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX)
Figure 8. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).

In addition, we plotted the visibility with 1-dimension at UV for confirmation as shown in figure 9. Figure 9 also shows the same result except the difference of flux both 0.75 Jy of Daejeon Correlator and 0.84 Jy of DiFXas plotted in above 2-dimensional map.

Flux density according to UV-distance. Left(Daejeon Correlator), Right(DiFX)
Figure 9. Flux density according to UV-distance. Left(Daejeon Correlator), Right(DiFX).

3.3 Analsys result for continuum(K11017A)

3.3.1 Spectrum shape of raw data

This experiment is to evaluate the performance of 2 frequency simultaneous observation and correlated in November 2012. 8 IF was assigned for 22 and 43 GHz respectively and we will report only 8 IFs for 22 GHz. We looked into the spectrum shape of POSSM after FITLD, MSORT, USUBA andINDXR task. The source is NRAO530, which is strong continuum source and shownin figure 10 for Yonsei-Ulsan baseline. Phase of 8 IFs is continuously changed.

Spectrum shape of 8-IFs continuum for Ulsan-Yonsei baseline. Left(Daejeon Correlator), Right(DiFX)
Figure 10. Spectrum shape of 8-IFs continuum for Ulsan-Yonsei baseline. Left(Daejeon Correlator), Right(DiFX).

3.3.2 Gain amplitude

We looked into the Gain amplitude variation of each sourcefor whole observation time after ACCOR as shown in figure 11. Daejeon Correlator and DiFX had same pattern. This observation was conducted forlong time with NRAO150 source, so the loss of 2~4sec at beginning of scan wasnot affected in correlation result.

Variation style of gain amplitude for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX)
Figure 11. Variation style of gain amplitude for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).

3.3.3 SNR, Delay, Rate after fringe fitting

Figure 12 shows the SNR, Delay and Rate for each baseline,respectively. The reference was set as Ulsan station. Integration time forfringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strongradio source in continuum sources and phase pattern is clearly remarkable. Forcomparison with DiFX, all of the patterns for eachresult are almost similar without any problems.

SNR, Delay, Rate after FRING for each IF of Tamna station Daejeon Correlator
SNR, Delay, Rate after FRING for each IF of Tamna station Daejeon Correlator
SNR, Delay, Rate after FRING for each IF of Tamna station DiFX
Figure 12. SNR, Delay, Rate after FRING for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).

3.3.4 Closure phase

By using previous fringe fitting result, Closure phase forNARO150 were obtained and showed in figure 13. We confirmed that closure phasewas about 5 degree by calculating with variation of phase for whole observationtime and by integrating 20~109 channels value of 1st IF for both Daejeon Correlator and DiFX.

Closure phase after fringe fitting
Figure 13. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).

3.3.5 Spectrum shape after fringe fitting and amplitude-cal

We draw again the spectrum in order to check phase residualby adopting the value of delay, rate after fringe fitting as shown in figure14. For more detail comparison, the range of phase was enlarged to 10 degreeand only 8 IFs were shown. Phase residual for spectrum of Daejeon Correlator and DiFX was almost same. And in order to compare the flux,amplitude calibration was also applied. We confirmed that flux of Daejeon Correlator hasabout 10% lower flux than DiFX as described inR11027B experimental result.

Spectrum shape after fringe fitting and amplitude calibration
Figure 14. Spectrum shape after fringe fitting and amplitude calibration. Left(Daejeon Correlator), Right(DiFX).

3.3.6 UV coverage, UV plot

UV coverage or UV plot methods are widely used to confirm how much made up for the UV during observation of NRAO150 source. UV plot is very useful to understand the flux distribution and structure of source. Figure 15 shows the UV coverage and UV plot for Daejeon Correlator result. From pattern of UV plot, we could understand that NRAO150 source is the bright spot with flat in KVN only baseline(less than about 30Mlambda).

The UV coverage (Left) and the amplitude of UV data(Right)
Figure 15. The UV coverage (Left) and the amplitude of UV data(Right).

3.3.7 Imaging performance

New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. And then 2-dimensional map was plotted using new result file as shown in figure 16. Flux for Daejeon Correlator is 6.34 Jy/beam, which is less than about 10% compared with 6.83 Jy/beam of DiFX, but the dynamic range is almost same value.

2-dimensional image map using Difmap
Figure 16. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).

3.4 Analysis result for spectral-line(R11027B)

3.4.1 Spectrum of raw data

We confirmed the spectrum shape of POSSM task aftercompletion through FITLD, MSORT, USUBA and INDXR tasks by separating only 9 IFof maser source in R11027B FITS file as shown in figure 17. The used masersource is SgrB2M, which shows strong emission line of H2O of 22 GHz. In case of9th IF, we conducted the comparison work for 3 kinds of correlator such asDaejeon Correlator, MTK and DiFX. As you can see in figure 17, theentire spectrum shape look like almost same aspect, but Daejeon Correlator has phaseconcentration in 0 degree at the beginning of bandwidth because of DC-likecomponent. These DC-like component and phase concentration are currentlydisappeared.

Spectrum shape of 9th IF of Yonsei and Ulsan baseline
Figure 17. Spectrum shape of 9th IF of Yonsei and Ulsan baseline.

3.4.2 Global fringe fitting(Calibrator)

Spectral-line observation such as maser source is occurredthe signal at very limited channel(within 10channels)per one maser spot. Therefore we should firstly apply the clock-offsetcompensation by performing the global fringe fitting according to the brightcalibrator source because it is difficult to get the delay in narrow frequencyrange. In this case, 3C345 source was used for global fringe fitting and figure18 shows the delay calculated by setting the reference as Tamnastation. Daejeon Correlator and MTK had almost same pattern of delay, because same delayparameter was used for correlation. In case of DiFX,it has more small delay value, and we think that this was caused by applyingthe more detail clock-offset compensation during correlation of DiFX. In figure 18, the delay of correlation result looksgood without any problem.

Fringe fitting result of 3C345 as calibrator source
Figure 18. Fringe fitting result of 3C345 as calibrator source.

3.4.3 SNR after fringe fitting(maser)

Fringe fitting was again conducted for maser source byapplying the delay value obtained after global fringe fitting. Although fringefitting for continuum was done in accordance with entire channel, fringefitting for spectral-line was done with only one channel which has peak value.In this figure, 197th channel has strong flux, so the fringe fitting was donewith this channel. Figure 19 shows the variation of SNR of maser source for eachstation based on Tamna station. Daejeon Correlator, MTK, and DiFX had almost same SNR and variation pattern also look like same.

result of SgrB2M as maser source
Figure 19. Fringe fitting result of SgrB2M as maser source, and this figure shows SNR based on Tamna station.

3.4.4 Spectrum shape after compensation of Dopper effect, amplitude calibration and fringe fitting

In VLBI observation, the Dopper effect such as the earth rotation or revolution is being affected at each station. Therefore, Dopper effect should becompensated at the data analysis stage. CVEL command of AIPS is used for the Dopper effect compensation. Amplitude calibration was alsoperformed so as to convert power level to real flux value for each station. Theresults were shown in figure 20. The Dopper effectwas successfully compensated for FITS file of 3 correlation result. The unitfor each result is Jy unit.

Spectrum shape after fringe fitting
Figure 20. Spectrum shape after fringe fitting, Dopper effect and amplitude calibration.

3.4.5 UV coverage, UV plot

It is very simple way to confirm the UV coverage in order toknow how UV was filled with during observation of SgrB2M maser source. It ishelpful to know the structure and distribution of flux of source asUV-distance. Figure 21 shows the UV coverage and UV plot of Daejeon Correlator correlationresult. From pattern of UV plot, SgrB2M has the structure of Gaussiandistribution at KVN+VERA 7 stations, and it looks like the bright spot withflat in KVN only baseline(less than about 30Mlambda).

UV coverage  UV plot
Figure 21. UV coverage(Left) and UV plot(Right) of SgrB2M. UV plot looks like Gaussian distribution, but only KVN baseline seems to be flat.

3.4.6 Imaging performance(single channel)

We conducted the imaging work with special channel usingmaser source as well as continuum source. In case of data processing stage forspectral-line, BPASS and CVEL task were additionally done. The result file wasgenerated by applying result tables for SgrB2M using SPLIT task. For comparison with DiFX, data for only KVN 3 stations was plottedin the same way of continuum source as shown in figure 22. Flux density for Daejeon Correlator is about 10% lower than DiFX, but dynamic rangeis almost same.

2-dimensional image map using AIPS Daejeon Correlator  2-dimensional image map using AIPS DiFX
Figure 22. 2-dimensional image map using AIPS. Left(Daejeon Correlator), Right(DiFX).

3.4.7 Imaging performance(multi channel)

In case of maser source, maser spots are existed withseveral tens and thousands in the uniform range. In general, imaging of masersource is performed as follows. Firstly we found each peak channel as refer tospectrum, and then the position of whole maser spots as indicated. Finally2-dimensional distribution map is plotted. In this experiment, in order toreproduce image, the position was obtained by imaging for several maser spot,and then we confirmed whether the position of maser spot for DiFX result was consistent or not. We conducted the multi-channel imaging work as shown in figure 23 so as to find the maser position, and confirmed that the maser spot for 264th, 269th, 274th, 280th, and 284th channel was detected.

multi-channel image a  multi-channel image b
Figure 23. Example of multi-channel image. Image result for each 261th-290th channel.

4. Two-layer problem had been solved

Daejeon Correlator had solved the recent issue, named the two-layer problem. This was caused by fault on the address set to data memory reading pointer in data serialize FPGA part. Resultant data mixtures across the sub-bands was found out as two-layer pattern on a time-power plot, as like as the left panel of Figure 24. The influence of two-layer problem is dependent on the power difference to the first sub-band(IF1),and the loss of visibility amplitude is estimated as less than ~3% for luckyless case.
After fixing that fault, the visibility outputs became stable with no such pattern.

Before: abnormal shape with two-layer pattern  After: ordinary (as expected) shape in Gain amplitude plot (KYS, r11027b)
Figure 24. (Left) Before: abnormal shape with two-layer pattern, (Right) After: ordinary (as expected) shape in Gain amplitude plot (KYS, r11027b)


KASI logo image

Korean VLBI Network (KVN)