Why is my sound lagging?_问答_开发者_运维开发者技术经验分享

I'm working on a system of applications for processing sound data. The first application simply reads from a microphone jack and sends the data to the next application. The main loop repeatedly performs this code:

0 : Globals.mySleep(waitTime); // tells the thread to sleep for the proper amount of time for a given data format  
1 : inputLine.read(buffer, 0, bufferSize); // reads sound data from the microphone jack into buffer
2 : if(connections.get(REGISTER) != null) { // if the next application is connected
3 :     DataSlice slice = new DataSlice(buffer, serialIDCounter++, getDeviceName()); // create a slice of data to send, containing the sound data
4 :     try{
5 :         connections.get(REGISTER).sendDataSlice(slice); // send the data to the next application. supposed to block until next application receives the data
6 :         connections.get(REGISTER).flush(); // make sure data gets sent
7 :     } catch (IOException e) {
8 :         // Stream has been broken. Shut Down
9 :         close();
10:     }
11: }

When I start the system, it is always several seconds behind. If I pause the system (GUI application tells the application following the input application to stop receiving data from the input application, so the input application should block at line 5 when paused), wait, then play again, the system lags additionally by however long I had just paused for. For example, if it started out with a 10-second lag, then paused for 5 seconds, and played again, it would then be lagging by 15 seconds.

This occurs when I run the program as a runnable jar file. It does not occur when I run it from Eclipse.

I've tested this on two computers, both running Ubuntu Linux 10.04 LTS. It occurs on one, not the other. Although on the other, I do get a whole different problem when I try to run it from Eclipse. Not sure what to make of this. If you would like some specs on the computers, I'd be happy to give them to you. Just tell me what specs you want and how to get them.

Could anyone tell me what might be causing the lag? Thanks.

--EDIT--

Per Andrew's suggestion, I created what I believe to be a SSCCE:

import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;

import javax.sound.sampled.*;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;

public class Main implements MouseListener{
    // Class that reads a signal from Line-in source and sends that signal
    // to either a recorder module or the signal-viewing pipeline
    public class PlayThread extends Thread {

        byte[] buffer = new byte[bufferSize];
        boolean playing = false;
        boolean connected = false;

        PlayThread() {}

        public void run() {
            while(true) {
                try {
                    sleep(waitTime);
                    inputLine.read(buffer, 0, bufferSize);
                    if(connected) {
                        while(!playing)
                            sleep(100);
                        int max = 0;
                        for(int i = 0; i < buffer.length; i++) {
                            if(Math.abs(buffer[i]) > max)
                                max = Math.abs(buffer[i]);
                        }
                        System.out.println("Max: " + max);
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }

        public void setPlaying(boolean playing) {
            this.playing = playing;
        }

        public void setConnected(boolean connected) {
            this.connected = connected;
        }
    }

    TargetDataLine inputLine;
    AudioFormat format;
    float sampleRate;
    int sampleSizeBits;
    int channels;
    int waitTime;
    int bufferSize;
    int slicesPerSecond;
    int windowSize = 512;
    PlayThread pThread;

    JFrame gui = new JFrame("Sound Lag");
    JPanel panel = new JPanel();
    JButton play = new JButton("Play"), pause = new JButton("Pause"),
            connect = new JButton("Connect"), disconnect = new JButton("Disconnect");

    Main() {
        sampleRate = 44100;
        sampleSizeBits = 16;
        channels = 2;
        bufferSize = (sampleSizeBits/8)*channels*windowSize;
        slicesPerSecond = (int) ((sampleRate/(float)channels)/(float)windowSize);
        waitTime = (int)((((1000f/sampleRate)/(float)sampleSizeBits)/2f)*8f*(float)bufferSize);

        play.addMouseListener(this);
        pause.addMouseListener(this);
        connect.addMouseListener(this);
        disconnect.addMouseListener(this);

        panel.add(play);
        panel.add(pause);
        panel.add(connect);
        panel.add(disconnect);
        gui.add(panel);
        gui.setVisible(true);
        gui.pack();
        gui.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    }

    public void read() {
        // Open line from line-in
        format = new AudioFormat(sampleRate, sampleSizeBits, channels, true, true);

        // Obtain and open the lines.
        inputLine = getTargetDataLine();

        pThread = new PlayThread();
        pThread.start();
    }

    private TargetDataLine getTargetDataLine() {
        try {
            DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
            for (Mixer.Info mi : AudioSystem.getMixerInfo()) {开发者_运维问答
                TargetDataLine dataline = null;
                try {
                    Mixer mixer = AudioSystem.getMixer(mi);
                    dataline = (TargetDataLine)mixer.getLine(info);
                    dataline.open(format);
                    dataline.start();
                    return dataline;
                }
                catch (Exception e) {}
                if (dataline != null)
                    try {
                        dataline.close();
                    }
                    catch (Exception e) {}
            }
        }
        catch (Exception e) {}
        return null;
    }

    public static void main(String[] args) {
        Main main = new Main();
        main.read();
    }

    @Override
    public void mouseClicked(MouseEvent arg0) {
        if(arg0.getSource() == play) {
            System.out.println("Playing");
            pThread.setPlaying(true);
        }
        else if(arg0.getSource() == pause) {
            System.out.println("Paused");
            pThread.setPlaying(false);
        }
        else if(arg0.getSource() == connect) {
            System.out.println("Connected");
            pThread.setConnected(true);
        }
        else if(arg0.getSource() == disconnect) {
            System.out.println("Disconnected");
            pThread.setConnected(false);
        }
    }

    @Override public void mouseEntered(MouseEvent arg0) {}
    @Override public void mouseExited(MouseEvent arg0) {}
    @Override public void mousePressed(MouseEvent arg0) {}
    @Override public void mouseReleased(MouseEvent arg0) {}
}

This code produces a window with four buttons in it: play, pause, connect, and disconnect. If you press play, it is as if the program were in "play" mode. If you click connect, it is as if the sound input application were connected to the next module.

To test, do the following:

Connect a sound device to your microphone jack (but don't play anything).

Create the runnable jar file from this code.

Run the file from a terminal.

Click "play".

Click "connect".

At this point, you should see a bunch of smaller numbers going down the terminal.

On your sound device, start playing sounds.

You should instantly start seeing bigger numbers in the terminal.

Stop playing sounds on sound device (should go back to smaller numbers in terminal).

Click "pause".

Wait 5 seconds.

Click "Play".

Start playing sound with audio device.

This is where the bug comes in. If I am running this code in Eclipse, I instantly get bigger numbers again. If I am just running the jar file, there is a 5-second delay, then I get the bigger numbers.

Any new thoughts?

It's fixed. Whenever I want to get the sound stream going (whenever I press play), I close the current stream and open a new one.

I didn't realize the TargetDataLine actually held a buffer of sound data that just gets picked from whenever the read method is called.

It looks like when I ran the application from Eclipse, it was using a different type of TargetDataLine than when I ran it as a runnable jar file. This was evidenced by a difference in size between the buffers. Although the size difference was only about a factor of 2, so I think the problem wasn't in the size of the buffer, but in something else having to do with the TargetDataLines that were fetched.

Oddly enough, removing Globals.mySleep(waitTime) worked in fixing the SSCCE, but not the real program it was supposed to represent.

I tried both draining and flushing the Line, instead of replacing it, but neither of these seemed to work, though I may have been using them wrong.

So the problem was: The DataLine's buffer was getting filled up and while the program was not playing, the buffer was not being emptied, so when it did start playing, it continued fetching data from the buffer at the usual play rate, causing it to lag behind.

The solution is: When the program starts playing, replace the DataLine.

--EDIT--

Further observation shows that when I ran from Eclipse, it seemed to be using a different JRE than when I ran as a jar file. I set the default java program to be java-6-sun instead of java-6-openjdk and it works fine from the jar file.

Also, I tried running the method of replacing the DataLine on a different computer. On this computer, I was getting a nasty break in the signal. It seemed to be taking longer to pull a new DataLine, so I decided that wasn't going to work. Now, I simply read from the DataLine at all times. I just don't send the signal anywhere if the system is paused.

I find the best thing to do in situations like this, where your code is slow but you dont know why is to use a profiler, http://www.quest.com/jprobe/software_download.aspx you can get a free trail of this java profiler and it will tell you line by line how much time is spent and how many times it is executed, you should be able to pinpoint exactly what is slowing you code down with this.

Hope this helps, Eamonn