Korean VLBI Network (KVN)

Status of Daejeon Correlator

1. Daejeon Correlator

Daejeon Correlator
Figure 1. Block diagram of Daejeon Correlator.

Figure 1 shows the conceptual block diagram of Daejeon Correlator. There are several VLBI data playbacks which will be used in our combined VLBI network, such as Mark5B, VERA2000, OCTADISK and optical fiber which shall be connected in near future. Some of them have the VSI-H compatible interface, but the others take the different interface for the data transmission. And they have also their own maximum data recording/playing back rates respectively. To absorb all of these differences and in homogeneity among these existing VLBI data playback systems, the Raw VLBI Data Buffer (RVDB) was introduced, which is a big data server with many large RAID disks and several kind of VLBI data interfaces. The VLBI Correlation Subsystem (VCS) shall receive the VLBI data from the RVDB system, shall calculate the correlation between the every possible pairs of data inputs with proper control parameters given from the monitor and control operation computer, and then shall dump the correlation results into the data archive system. Data archive system, is also a kind of data server, which is used to capture the correlated data output from the VCS and to save them as a structured file system. Finally there is also the correlator control and operation software for overall system.

1.1 Playback System

KVN is now using the Mark5B system for recording and playback the observed data. KASI participated in Mark5B development with Haystack Observatory as a member of international consortium. It can support the VSI compatible and RAID-based HDD storage system. Recording and playing back speed is 1024 Mbps. DIR2000 is widely used in VERA for recording and playback with 1024 Mbps. Recently the manufacturer of DIR2000 had been stopped to manufacture, so NAOJ developed new playing back system named VERA2000, which is modified by DIR1000H system for only supporting playing back. For improving observation and correlation efficiency, VERA uses OCTADISK system for KaVA observations from 2014. So Daejeon Correlator also adopted OCTADISK system, which has the same functions of OCTADDB installed in RVDB (Refer 1.2).

1.2 RVDB system

RVDB system, which is developed by NAOJ, consists of Data Input Output (DIO) (currently named OCTAVIA), 10 GbE switch, and Disk Data Buffer (DDB) (currently named OCTADDB). It is able to record the data with maximum 2048 Mbps, and can play back the observed data to the correlator with nominal 2048 Mbps. So, it has a function such as 2048 Mbps recorder and playback at the same time. As shown in Figure 1, the different types of playback systems are used in Daejeon Correlator. So, the purpose of the RVDB system is that it is able to adjust the data format and easily synchronize the data during playback and maintain the buffering between recorder speed and correlation speed.

1.3 VLBI Correlation Subsystem

The main specification of VCS is described in Table 1. The VCS has a capability to process the total 120 cross-correlation and 16 auto-correlation intended for maximum 16 stations, maximum 8192 Mbps (4-streams x 1 Gsps/2-bit/64 MHz clock) input data rates per station. The design architecture for correlator is FX-based and it will use the variable length of FFT (Fast Fourier Transform) to maintain the 0.05 km/sec resolution of velocity at 22 GHz. The maximum delay is ±36,000 km and maximum fringe tracking is 1.075 kHz. The number of frequency channel per correlation output is 8,192.

Table 1. Specification of Daejeon Correlator.
# of antennas 16
# of inputs/antenna 4 bands(4Fx1P, 2Fx2P, 1Fx2P+2Fx1P)
Max. # of corr./input 120 cross + 16 auto
Sub-array 2 case(12+4, 8+8)
Bandwidth 512 MHz
Sampling, Digitization 1 Gsps by 2bit/sample
Maximum data rate/antenna 2 Gbps VSI-H (32parallels, 64 MHz clock)
Maximum delay compensation ±36,000km
Maximum fringe tracking 1.075 kHz
Design architecture FX-type with FPGA
FFT word length 16+16 bits fixed point for real, imaginary
Integration time 25.6 msec ~ 10.24 sec
Data output channels 8192 channels
Data output rate Maximum 1.4GB/sec at 25.6msec integration time

1.4 Correlated VLBI Data Buffer (CVDB) (old name : Data Archive System)

The basic architecture is a CPU cluster connected with infiniband. For KaVA, the first phase data archive system with about 119TB capacity has been implemented. It has four 10 Gbit Ethernet input port to connect with VCS output and one 10 GbE Ethernet port is connected with data file system for sharing the disk with each other. We have a plan to increase the system capacity for support the EAVN in near future. CODA file system is used in data archive system for making the file system from correlated raw data, which is revised with ccCODA 2.0 library used in Mitaka FX correlator with some modification. New CODA files system based on the ccCODA 2.1 library was installed. Figure 2 shows the view of Daejeon Correlator installed at Korea-Japan Correlation Center (KJCC) located in KASI, Daejeon, Korea.

Daejeon Correlator installed at Korea-Japan Correlation Center
Figure 2. Daejeon Correlator installed at Korea-Japan Correlation Center.

2. Correlation Processing

2.1 Correlation mode and Integration time

Daejeon Correlator is currently able to support the following correlation modes.

Table 2. Correlation Modes in Daejeon Correlator
Obs. Mode Total Data Rate Bandwidth/sub-band # of sub-bands Minimum Accum. Time # of Freq. Channels/sub-band
C5 1024 Mbps 16 MHz 16 1.6384 sec 8192
C4 1024 Mbps 32 MHz 8 0.8192 sec 8192
C3 1024 Mbps 64 MHz 4 0.4096 sec 8192
C2 1024 Mbps 128 MHz 2 0.2048 sec 8192
C1 1024 Mbps 256 MHz 1 0.1024 sec 8192
W1 2048 Mbps 512 MHz 1 0.0512 sec 8192
W2 4096 Mbps 512 MHz 2 0.0512 sec 8192
W4 8192 Mbps 512 MHz 4 0.0512 sec 8192

2.2 CODA/FITS

Daejeon Correlator will basically support following frequency channel for preparing FITS file.

  • Basic output channel of correlator : 8192 frequency channel
  • Continuum : 128 frequency channel(64 channel integration)
  • Spectral line : 512 frequency channel(16 channel integration)

2.3 FITS delivery

KJCC will deliver FITS file to PI by using FTP server or mobile disk.

2.4 Archiving policy

KJCC will organize the archiving policy for observation data, CODA and FITS file as below.

  • Observation data : basically 2 months, and then it will be released.
  • CODA : if correlated data will be used for astrometry or geodesy, it will be permanently stored at CODA server.
    Otherwise, the correlated raw data and CODA file system will be deleted after receiving the response from PI.
  • FITS : it will be permanently archived at Archiving server.

3. Performance of Daejeon Correlator (Continuum/Spectral-line)

3.1 Data Analysis (@AIPS)

In order to verify the performance of Daejeon correlator, data analysis was conducted. For the comparison Daejeon Correlator, Mitaka FX and DiFX correlator were used for data correlation. Observation experiments were listed in Table 3. We confirmed that results of Daejeon Correlator had consistent and correct value. Here we will summarize the results for R11027B and K11017A.
Procedure of data analysis was performed as below. And the same parameter was used for all correlation result during analysis. Analysis for spectral-line was adopted with following procedure, but analysis for continuum was excluded for 9 and 10 stages.

Table 3. Observation experiments list.
Experiment Object Source Recorder Comparison correlator
R11027B Daejeon Correlator evaluation Continuum, Spectral line DIR2K, Mark5B MTK FX, DiFX
R11026A Long time phase monitoring Continuum DIR1K, DIR2K, Mark5B MTK FX
K11017A 2 frequency simultaneous observation Continuum Mark5B DiFX
K12098C 4 frequency simultaneous observation Continuum Mark5B DiFX

3.2 Analysis result for continuum (R11027B)

3.2.1 Spectrum of raw data

Figure 3 shows the spectrum shape after completion of FITLD, MSORT, USUBA and INDXR task based on FITS file generated by Daejeon Correlator and DiFX. Source is 3C454.3 for continuum with Yonsei and Ulsan baseline. Phase for all of 16 IF are continuously changed. The phase slope for DiFX is lower than Daejeon Correlator, but we think this is caused by adopting the clock-offset precisely while DiFX correlation. After fringe fitting, these values will be deleted and there is no serious problem in current phase slope.

Spectrum shape of 16IF continuum for Yonsei-Ulsan baseline
Figure 3. Spectrum shape of 16IF continuum for Yonsei-Ulsan baseline.

3.2.2 Gain amplitude

We looked into the variation of GAIN for each source for whole observation time after ACCOR. And then in figure 4, gain amplitude was continuously stable for source with long time integration. But in case of source with short integration time, the gain amplitude pattern for Daejeon Correlator was unstable for time range(between 22:00 and 03:00 for SgrA, Sgr2M) compared with DiFX. This phenomenon was appeared in only KVN station, this is caused with data synchronization while playing back. This phenomenon only occurred beginning of scan about 2~4sec, and the other time range is stable. We recommended the continuous recording of Mark5B while observation.

Variation style of gain amplitude for all stations with 9th IF. Left(Daejeon Correlator)  Variation style of gain amplitude for all stations with 9th IF. Center(MTK FX), Right(DiFX)
Figure 4. Variation style of gain amplitude for all stations with 9th IF. Left(Daejeon Correlator), Center(MTK FX), Right(DiFX).

3.2.3 SNR, Delay, Rate after fringe fitting

Figure 5 shows the SNR, Delay and Rate for each baseline, respectively. The reference was set as Ulsan station. Integration time for fringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strong radio source in continuum sources and phase pattern is clearly remarkable. For comparison with DiFX, all of the patterns for each result are almost similar without any problems.

 SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator)   SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator)   SNR, Delay, Rate after FRING for 1st IF of Yonsei. Bottom(DiFX)
Figure 5. SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator), Bottom(DiFX).

3.2.4 Closure phase

By using previous fringe fitting result, Closure phase for 3C454.3 were obtained and showed in figure 6. We confirmed that closure phase was existed within about 5 degree by calculating with variation of phase for whole observation time and by integrating 50~64 channels value of 1st IF for both Daejeon Correlator and DiFX.

Closure phase after fringe fitting. Upper(Daejeon Correlator)  Closure phase after fringe fitting. Bottom(DiFX)
Figure 6. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).

3.2.5 Spectrum shape after fringe fitting

Figure 7 shows the spectrum shape after fringe fitting. We draw again the spectrum in order to check phase residual by adopting the value of delay, rate after fringe fitting. For more detail comparison, the range of phase was enlarged to 10 degree and only 8 IFs were shown. Phase residual for spectrum of Daejeon Correlator and DiFX was almost same with 0 value.

Spectrum shape after fringe fitting. Left(Daejeon Correlator)  Spectrum shape after fringe fitting. Right(DiFX)
Figure 7. Spectrum shape after fringe fitting. Left(Daejeon Correlator), Right(DiFX).

3.2.6 Imaging performance

New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. The 2-dimensional image map was generated using new result file and is shown in figure 8. In figure 8, only KVN 3 stations were used for comparison to DiFX. Flux for Daejeon Correlator is 0.75 Jy/beam, which is less than about 12% compared with 0.84 Jy/beam of DiFX, but the dynamic range is almost same value.

2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX)
Figure 8. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).

In addition, we plotted the visibility with 1-dimension at UV for confirmation as shown in figure 9. Figure 9 also shows the same result except the difference of flux both 0.75 Jy of Daejeon Correlator and 0.84 Jy of DiFX as plotted in above 2-dimensional map.

Flux density according to UV-distance. Left(Daejeon Correlator), Right(DiFX)
Figure 9. Flux density according to UV-distance. Left(Daejeon Correlator), Right(DiFX).

3.3 Analsys result for continuum(K11017A)

3.3.1 Spectrum shape of raw data

This experiment is to evaluate the performance of 2 frequency simultaneous observation and correlated in November 2012. 8 IF was assigned for 22 and 43 GHz respectively and we will report only 8 IFs for 22 GHz. We looked into the spectrum shape of POSSM after FITLD, MSORT, USUBA and INDXR task. The source is NRAO530, which is strong continuum source and shown in figure 10 for Yonsei-Ulsan baseline. Phase of 8 IFs is continuously changed.

Spectrum shape of 8-IFs continuum for Ulsan-Yonsei baseline. 
Left(Daejeon Correlator), Right(DiFX)
Figure 10. Spectrum shape of 8-IFs continuum for Ulsan-Yonsei baseline. Left(Daejeon Correlator), Right(DiFX).

3.3.2 Gain amplitude

We looked into the Gain amplitude variation of each source for whole observation time after ACCOR as shown in figure 11. Daejeon Correlator and DiFX had same pattern. This observation was conducted for long time with NRAO150 source, so the loss of 2~4sec at beginning of scan was not affected in correlation result.

Variation style of gain amplitude for each IF of Tamna station. 
Left(Daejeon Correlator), Right(DiFX)
Figure 11. Variation style of gain amplitude for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).

3.3.3 SNR, Delay, Rate after fringe fitting

Figure 12 shows the SNR, Delay and Rate for each baseline, respectively. The reference was set as Ulsan station. Integration time for fringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strong radio source in continuum sources and phase pattern is clearly remarkable. For comparison with DiFX, all of the patterns for each result are almost similar without any problems.

SNR, Delay, Rate after FRING for each IF of Tamna station Daejeon Correlator
SNR, Delay, Rate after FRING for each IF of Tamna station Daejeon Correlator
SNR, Delay, Rate after FRING for each IF of Tamna station DiFX
Figure 12. SNR, Delay, Rate after FRING for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).

3.3.4 Closure phase

By using previous fringe fitting result, Closure phase for NARO150 were obtained and showed in figure 13. We confirmed that closure phase was about 5 degree by calculating with variation of phase for whole observation time and by integrating 20~109 channels value of 1st IF for both Daejeon Correlator and DiFX.

Closure phase after fringe fitting
Figure 13. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).

3.3.5 Spectrum shape after fringe fitting and amplitude-cal

We draw again the spectrum in order to check phase residual by adopting the value of delay, rate after fringe fitting as shown in figure 14. For more detail comparison, the range of phase was enlarged to 10 degree and only 8 IFs were shown. Phase residual for spectrum of Daejeon Correlator and DiFX was almost same. And in order to compare the flux, amplitude calibration was also applied. We confirmed that flux of Daejeon Correlator has about 10% lower flux than DiFX as described in R11027B experimental result.

Spectrum shape after fringe fitting and amplitude calibration
Figure 14. Spectrum shape after fringe fitting and amplitude calibration. Left(Daejeon Correlator), Right(DiFX).

3.3.6 UV coverage, UV plot

UV coverage or UV plot methods are widely used to confirm how much made up for the UV during observation of NRAO150 source. UV plot is very useful to understand the flux distribution and structure of source. Figure 15 shows the UV coverage and UV plot for Daejeon Correlator result. From pattern of UV plot, we could understand that NRAO150 source is the bright spot with flat in KVN only baseline(less than about 30Mlambda).

The UV coverage (Left) and the amplitude of UV data(Right)
Figure 15. The UV coverage (Left) and the amplitude of UV data(Right).

3.3.7 Imaging performance

New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. And then 2-dimensional map was plotted using new result file as shown in figure 16. Flux for Daejeon Correlator is 6.34 Jy/beam, which is less than about 10% compared with 6.83 Jy/beam of DiFX, but the dynamic range is almost same value.

2-dimensional image map using Difmap
Figure 16. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).

3.4 Analysis result for spectral-line(R11027B)

3.4.1 Spectrum of raw data

We confirmed the spectrum shape of POSSM task after completion through FITLD, MSORT, USUBA and INDXR tasks by separating only 9 IF of maser source in R11027B FITS file as shown in figure 17. The used maser source is SgrB2M, which shows strong emission line of H2O of 22 GHz. In case of 9th IF, we conducted the comparison work for 3 kinds of correlator such as Daejeon Correlator, MTK and DiFX. As you can see in figure 17, the entire spectrum shape look like almost same aspect, but Daejeon Correlator has phase concentration in 0 degree at the beginning of bandwidth because of DC-like component. These DC-like component and phase concentration are currently disappeared.

Spectrum shape of 9th IF of Yonsei and Ulsan baseline
Figure 17. Spectrum shape of 9th IF of Yonsei and Ulsan baseline.

3.4.2 Global fringe fitting(Calibrator)

Spectral-line observation such as maser source is occurred the signal at very limited channel(within 10channels) per one maser spot. Therefore we should firstly apply the clock-offset compensation by performing the global fringe fitting according to the bright calibrator source because it is difficult to get the delay in narrow frequency range. In this case, 3C345 source was used for global fringe fitting and figure 18 shows the delay calculated by setting the reference as Tamna station. Daejeon Correlator and MTK had almost same pattern of delay, because same delay parameter was used for correlation. In case of DiFX, it has more small delay value, and we think that this was caused by applying the more detail clock-offset compensation during correlation of DiFX. In figure 18, the delay of correlation result looks good without any problem.

Fringe fitting result of 3C345 as calibrator source
Figure 18. Fringe fitting result of 3C345 as calibrator source.

3.4.3 SNR after fringe fitting(maser)

Fringe fitting was again conducted for maser source by applying the delay value obtained after global fringe fitting. Although fringe fitting for continuum was done in accordance with entire channel, fringe fitting for spectral-line was done with only one channel which has peak value. In this figure, 197th channel has strong flux, so the fringe fitting was done with this channel. Figure 19 shows the variation of SNR of maser source for each station based on Tamna station. Daejeon Correlator, MTK, and DiFX had almost same SNR and variation pattern also look like same.

result of SgrB2M as maser source
Figure 19. Fringe fitting result of SgrB2M as maser source, and this figure shows SNR based on Tamna station.

3.4.4 Spectrum shape after compensation of Dopper effect, amplitude calibration and fringe fitting

In VLBI observation, the Dopper effect such as the earth rotation or revolution is being affected at each station. Therefore, Dopper effect should be compensated at the data analysis stage. CVEL command of AIPS is used for the Dopper effect compensation. Amplitude calibration was also performed so as to convert power level to real flux value for each station. The results were shown in figure 20. The Dopper effect was successfully compensated for FITS file of 3 correlation result. The unit for each result is Jy unit.

Spectrum shape after fringe fitting
Figure 20. Spectrum shape after fringe fitting, Dopper effect and amplitude calibration.

3.4.5 UV coverage, UV plot

It is very simple way to confirm the UV coverage in order to know how UV was filled with during observation of SgrB2M maser source. It is helpful to know the structure and distribution of flux of source as UV-distance. Figure 21 shows the UV coverage and UV plot of Daejeon Correlator correlation result. From pattern of UV plot, SgrB2M has the structure of Gaussian distribution at KVN+VERA 7 stations, and it looks like the bright spot with flat in KVN only baseline(less than about 30Mlambda).

UV coverage  UV plot
Figure 21. UV coverage(Left) and UV plot(Right) of SgrB2M. UV plot looks like Gaussian distribution, but only KVN baseline seems to be flat.

3.4.6 Imaging performance(single channel)

We conducted the imaging work with special channel using maser source as well as continuum source. In case of data processing stage for spectral-line, BPASS and CVEL task were additionally done. The result file was generated by applying result tables for SgrB2M using SPLIT task. For comparison with DiFX, data for only KVN 3 stations was plotted in the same way of continuum source as shown in figure 22. Flux density for Daejeon Correlator is about 10% lower than DiFX, but dynamic range is almost same.

2-dimensional image map using AIPS Daejeon Correlator  2-dimensional image map using AIPS DiFX
Figure 22. 2-dimensional image map using AIPS. Left(Daejeon Correlator), Right(DiFX).

3.4.7 Imaging performance(multi channel)

In case of maser source, maser spots are existed with several tens and thousands in the uniform range. In general, imaging of maser source is performed as follows. Firstly we found each peak channel as refer to spectrum, and then the position of whole maser spots as indicated. Finally 2-dimensional distribution map is plotted. In this experiment, in order to reproduce image, the position was obtained by imaging for several maser spot, and then we confirmed whether the position of maser spot for DiFX result was consistent or not. We conducted the multi-channel imaging work as shown in figure 23 so as to find the maser position, and confirmed that the maser spot for 264th, 269th, 274th, 280th, and 284th channel was detected.

multi-channel image a  multi-channel image b
Figure 23. Example of multi-channel image. Image result for each 261th-290th channel.

4. Two-layer problem had been solved

Daejeon Correlator had solved the recent issue, named the two-layer problem. This was caused by fault on the address set to data memory reading pointer in data serialize FPGA part. Resultant data mixtures across the sub-bands was found out as two-layer pattern on a time-power plot, as like as the left panel of Figure 24. The influence of two-layer problem is dependent on the power difference to the first sub-band(IF1), and the loss of visibility amplitude is estimated as less than ~3% for luckyless case.
After fixing that fault, the visibility outputs became stable with no such pattern.

Before: abnormal shape with two-layer pattern  After: ordinary (as expected) shape in Gain amplitude plot (KYS, r11027b)
Figure 24. (Left) Before: abnormal shape with two-layer pattern, (Right) After: ordinary (as expected) shape in Gain amplitude plot (KYS, r11027b)


KASI logo image

Korean VLBI Network (KVN)