Using VLC imem to play h264 video file from memory but getting error "main stream error: cannot prefill buffer"

Question

Using VLC imem to play h264 video file from memory but getting error "main stream error: cannot prefill buffer"

I have an h264 video file that is loaded into memory and I am trying to play it with an imem using the "imem-cat = 4" parameter so that vlc will use the accessor to show the video and vlc starts up and gets my imem parameters successfully:

[0x7f38a0000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7fff5b4a9430), cookie("IMEM")

This category also means that I do not need to provide DTS and PTS. The imem module for VLC is poorly documented, but I found hints in several places, eg.

https://forum.videolan.org/viewtopic.php?t=111917

https://forum.videolan.org/viewtopic.php?f=32&t=93842

Play video using libVLC from memory in python

My imem-get function just sets the buffer pointer to the video data on the first call, returning 0, on any subsequent calls it returns 1 to indicate that there is no more data:

int MyImemGetCallback (void *data,
                   const char *cookie,
                   int64_t *dts,
                   int64_t *pts,
                   unsigned *flags,
                   size_t * bufferSize,
                   void ** buffer)
{

ImemData* imem = (ImemData*)data;
cookie = imem->cookieString;

if(imem == NULL || imem->allBuffered==true) //indicate all data has been get()ted
    return 1;

*buffer = (void*) imem->video;
bufferSize = (size_t*) &(imem->bytes);
imem->allBuffered=true;

return 0;
}

Unfortunately, after the first call, I get the following errors:

[0x189cb18] main input debug: Creating an input for 'imem://'
[0x189cb18] main input debug: using timeshift granularity of 50 MiB, in path '/tmp'
[0x189cb18] main input debug: `imem://' gives access `imem' demux `' path `'
[0x189cb18] main input debug: creating demux: access='imem' demux='' location='' file='(null)'
[0x7f2808000e28] main demux debug: looking for access_demux module matching "imem": 20 candidates
[0x7f2808000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808000e28] main demux debug: no access_demux modules matched
[0x189cb18] main input debug: creating access 'imem' location='', path='(null)'
[0x7f2808001958] main access debug: looking for access module matching "imem": 25 candidates
[0x7f2808001958] access_imem access debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808001958] main access debug: using access module "access_imem"
[0x7f2808000e28] main stream debug: Using block method for AStream*
[0x7f2808000e28] main stream debug: starting pre-buffering
[0x7f2808000e28] main stream error: cannot pre fill buffer
[0x7f2808001958] main access debug: removing module "access_imem"
[0x189cb18] main input warning: cannot create a stream_t from access
[0x17d7298] main libvlc debug: removing all interfaces
[0x17d7298] main libvlc debug: exiting
[0x17d7298] main libvlc debug: no exit handler
[0x17d7298] main libvlc debug: removing stats

For some reason, it seems that vlc cannot access the video data, but the error messages are not very helpful and usually refer to network streams and not memory locations.

Has anyone successfully used imem this way, or have any ideas as to what the problem might be? The video plays fine in VLC from disc. Thanks for any help.

Edit

It looks like the element's interface might not support the game this way. However, libVLC provides libvlc_media_t and livblc_media_new_callbacks, which may allow me to achieve what I want. I will lecture if I work.

+3

c ++ linux video h.264 vlc

Flakker 06 jul. 15 at 16:10

source to share

2 answers

Take a look at my Qt example below, it works. I didn't really want to read the entire file and store it in RAM. So I implemented MediaDescriptor because I will be implementing decryption logic in it to read from encrypted files. By the way, I used ready-made libvlc 3.0.6 x64 libraries, it looks very good.

MediaDescriptor.h

#pragma once

#include <QObject>
#include <fstream>

class MediaDescriptor : public QObject
{
    Q_OBJECT

public:
    MediaDescriptor(QString mediaFilePath);
    ~MediaDescriptor();

    bool tryOpen();
    uint64_t getMediaLength();
    uint64_t getMediaBytes(unsigned char *buffer, uint64_t length);
    void setSeek(uint64_t seek);

private:
    QString m_mediaFilePath;
    std::ifstream *m_mediaFile;
    uint64_t m_mediaLength;
    uint64_t m_seek;
};

MediaDescriptor.cpp

#include "MediaDescriptor.h"

MediaDescriptor::MediaDescriptor(QString mediaFilePath)
    : m_mediaFilePath(mediaFilePath), m_mediaFile(nullptr), m_mediaLength(0), m_seek(0)
{
}

MediaDescriptor::~MediaDescriptor()
{
    if (m_mediaFile)
    {
        m_mediaFile->close();
        delete m_mediaFile;
    }
}

bool MediaDescriptor::tryOpen()
{
    m_mediaFile = new std::ifstream(m_mediaFilePath.toStdString().c_str(), std::ios::binary | std::ios::ate);

    if (m_mediaFile->is_open())
    {
        m_mediaFile->seekg(0, m_mediaFile->end);
        m_mediaLength = m_mediaFile->tellg();
        return true;
    }

    delete m_mediaFile;
    return false;
}

uint64_t MediaDescriptor::getMediaLength()
{
    return m_mediaLength;
}

uint64_t MediaDescriptor::getMediaBytes(unsigned char *buffer, uint64_t length)
{
    // to do: decrytion logic
    if (m_mediaFile->is_open())
    {
        uint64_t len = length;
        if (m_seek + len > m_mediaLength)
            len = (size_t)(m_mediaLength - m_seek);

        if (len > 0)
        {
            m_mediaFile->seekg(m_seek);
            m_mediaFile->read((char*)&buffer[0], len);
            m_seek += len;
        }

        return len;
    }

    return -1;
}

void MediaDescriptor::setSeek(uint64_t seek)
{
    m_seek = seek;
}

VLCHelper.h

#pragma once

#include <QObject>
#include <QWidget>
#include <QTime>
#include <mutex>
#include "vlc/vlc.h"
#include "MediaDescriptor.h"

class VLCHelper : public QObject
{
    Q_OBJECT

public:
    ~VLCHelper();
    static VLCHelper& getInstance();

    int vlcMediaOpenCallback(void* opaque, void** datap, uint64_t* sizep);
    int vlcMediaReadCallback(void *opaque, unsigned char* buf, size_t len);
    int vlcMediaSeekCallback(void *opaque, uint64_t offset);
    void vlcMediaCloseCallback(void *opaque);

    void initMedia(MediaDescriptor &mediaDescriptor, QWidget *output = nullptr, int volume = 100, bool repeat = false);
    void destroyMedia();
    bool isMediaValid();

    Q_INVOKABLE void playPauseMedia(bool play);
    bool isMediaPlaying();
    Q_INVOKABLE void stopMedia();

    void setRepeatMedia(bool repeat);
    bool getRepeatMedia();

    void setMediaPosition(float position);
    float getMediaPosition();

    QTime getMediaTime();
    QTime getMediaTotalTime();

    void setMediaVolume(int volume);
    int getMediaVolume();

signals:
    void mediaEOFReached();
    void error(QString error);

private:
    VLCHelper();

    std::mutex m_callbackMutex;
    libvlc_instance_t *m_vlcInstance;
    libvlc_media_t *m_vlcMedia;
    libvlc_media_player_t *m_vlcMediaPlayer;
    bool m_repeat;
    bool m_stopRequested;
    MediaDescriptor *m_mediaDescriptor;
    QWidget *m_output;
};

VLCHelper.cpp

#include "VLCHelper.h"

#pragma region Callback Wrappers
extern "C" int vlcMediaOpenCallbackGateway(void* opaque, void** datap, uint64_t* sizep)
{
    return VLCHelper::getInstance().vlcMediaOpenCallback(opaque, datap, sizep);
}

extern "C" int vlcMediaReadCallbackGateway(void *opaque, unsigned char* buf, size_t len)
{
    return VLCHelper::getInstance().vlcMediaReadCallback(opaque, buf, len);
}

extern "C" int vlcMediaSeekCallbackGateway(void *opaque, uint64_t offset)
{
    return VLCHelper::getInstance().vlcMediaSeekCallback(opaque, offset);
}

extern "C" void vlcMediaCloseCallbackGateway(void *opaque)
{
    VLCHelper::getInstance().vlcMediaCloseCallback(opaque);
}
#pragma endregion

VLCHelper::VLCHelper()
    : QObject(nullptr),
    m_vlcInstance(nullptr),
    m_vlcMedia(nullptr),
    m_vlcMediaPlayer(nullptr),
    m_repeat(false),
    m_stopRequested(false)
{
}

VLCHelper::~VLCHelper()
{
}

VLCHelper& VLCHelper::getInstance()
{
    static VLCHelper ins;
    return ins;
}

void VLCHelper::initMedia(MediaDescriptor &mediaDescriptor, QWidget *output, int volume, bool repeat)
{
    m_mediaDescriptor = &mediaDescriptor;
    m_output = output;
    m_repeat = repeat;

    m_vlcInstance = libvlc_new(0, NULL);

    m_vlcMedia = libvlc_media_new_callbacks(
        m_vlcInstance,
        vlcMediaOpenCallbackGateway,
        vlcMediaReadCallbackGateway,
        vlcMediaSeekCallbackGateway,
        vlcMediaCloseCallbackGateway,
        0
    );

    m_vlcMediaPlayer = libvlc_media_player_new_from_media(m_vlcMedia);

#if defined(Q_OS_WIN)
    libvlc_media_player_set_hwnd(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#elif defined(Q_OS_MAC)
    libvlc_media_player_set_nsobject(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#elif defined(Q_OS_LINUX)
    libvlc_media_player_set_xwindow(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#endif

    libvlc_audio_set_volume(m_vlcMediaPlayer, volume);

    m_mediaDescriptor->setSeek(0);
}

void VLCHelper::destroyMedia()
{
    if (!m_vlcInstance)
        return;

    if (m_vlcMediaPlayer)
    {
        libvlc_media_player_stop(m_vlcMediaPlayer);
        libvlc_media_player_release(m_vlcMediaPlayer);
        m_vlcMediaPlayer = nullptr;
    }

    libvlc_release(m_vlcInstance);
    m_vlcInstance = nullptr;
}

bool VLCHelper::isMediaValid()
{
    return m_vlcInstance && m_vlcMedia && m_vlcMediaPlayer;
}

void VLCHelper::playPauseMedia(bool play)
{
    m_stopRequested = false;

    if (isMediaValid())
        play ? libvlc_media_player_play(m_vlcMediaPlayer) : libvlc_media_player_pause(m_vlcMediaPlayer);
    else
        emit error("TO DO");
}

bool VLCHelper::isMediaPlaying()
{
    if (isMediaValid())
        return libvlc_media_player_is_playing(m_vlcMediaPlayer);

    return false;
}

void VLCHelper::stopMedia()
{
    m_stopRequested = true;

    if (isMediaValid())
        libvlc_media_player_stop(m_vlcMediaPlayer);
    else
        emit error("TO DO");
}

void VLCHelper::setRepeatMedia(bool repeat)
{
    m_repeat = repeat;
}

bool VLCHelper::getRepeatMedia()
{
    return m_repeat;
}

void VLCHelper::setMediaPosition(float position)
{
    if (isMediaValid())
        libvlc_media_player_set_position(m_vlcMediaPlayer, position);
    else
        emit error("TO DO");
}

float VLCHelper::getMediaPosition()
{
    if (isMediaValid())
        return libvlc_media_player_get_position(m_vlcMediaPlayer);
    else
        emit error("TO DO");

    return -1.0;
}

QTime VLCHelper::getMediaTime()
{
    if (isMediaValid())
        return QTime::fromMSecsSinceStartOfDay((int)libvlc_media_player_get_time(m_vlcMediaPlayer));
    else
        emit error("TO DO");

    return QTime();
}

QTime VLCHelper::getMediaTotalTime()
{
    if (isMediaValid())
        return QTime::fromMSecsSinceStartOfDay((int)libvlc_media_player_get_length(m_vlcMediaPlayer));
    else
        emit error("TO DO");

    return QTime();
}

void VLCHelper::setMediaVolume(int volume)
{
    if (isMediaValid())
        libvlc_audio_set_volume(m_vlcMediaPlayer, volume);
    else
        emit error("TO DO");
}

int VLCHelper::getMediaVolume()
{
    if (isMediaValid())
        return libvlc_audio_get_volume(m_vlcMediaPlayer);
    else
        emit error("TO DO");

    return -1;
}

int VLCHelper::vlcMediaOpenCallback(void* opaque, void** datap, uint64_t* sizep)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    // optional, if comment out libvlc will trigger the 'vlcMediaReadCallback' as more often but for less buffer length
    *sizep = m_mediaDescriptor->getMediaLength();

    return 0;
}

int VLCHelper::vlcMediaReadCallback(void *opaque, unsigned char* buf, size_t len)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    return m_mediaDescriptor->getMediaBytes(buf, len);
}

int VLCHelper::vlcMediaSeekCallback(void *opaque, uint64_t offset)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    // optional, but important for some media types which holds meta data end of the file, for example: .mp4
    m_mediaDescriptor->setSeek(offset);

    return 0;
}

void VLCHelper::vlcMediaCloseCallback(void *opaque)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    m_mediaDescriptor->setSeek(0);

    if (!m_stopRequested)
    {
        emit mediaEOFReached();
        QMetaObject::invokeMethod(&getInstance(), "stopMedia");

        if(m_repeat)
            QMetaObject::invokeMethod(&getInstance(), "playPauseMedia", Q_ARG(bool, true));
    }
}

And use in the player widget:

    VLCHelper::getInstance().initMedia(*m_mediaDescriptor, ui.frame_video);
    connect(&VLCHelper::getInstance(), SIGNAL(mediaEOFReached()), this, SLOT(vlcMediaEOFReached()));
    connect(&VLCHelper::getInstance(), SIGNAL(error(QString)), this, SLOT(vlcError(QString)));

...

void PlayerWidget::on_pushButton_media_play_pause_clicked()
{
      VLCHelper::getInstance().playPauseMedia(!VLCHelper::getInstance().isMediaPlaying());
}

void PlayerWidget::on_pushButton_media_stop_clicked()
{
    VLCHelper::getInstance().stopMedia();
}

void PlayerWidget::timer_timeout()
{
    bool isValid = VLCHelper::getInstance().isMediaValid();

    if (isValid)
    {
        if (VLCHelper::getInstance().isMediaPlaying())
        {
            // update media position
            ui.horizontalSlider_media_position->blockSignals(true);
            ui.horizontalSlider_media_position->setValue((int)(VLCHelper::getInstance().getMediaPosition() * 1000.0f));
            ui.horizontalSlider_media_position->blockSignals(false);

            // update media volume
            ui.horizontalSlider_media_volume->blockSignals(true);
            ui.horizontalSlider_media_volume->setValue(VLCHelper::getInstance().getMediaVolume());
            ui.horizontalSlider_media_volume->blockSignals(false);

            // update media time
            ui.label_media_time->setText(VLCHelper::getInstance().getMediaTime().toString());

            // update media total time
            ui.label_media_time_total->setText(VLCHelper::getInstance().getMediaTotalTime().toString());
        }
    }

    ui.horizontalSlider_media_volume->setEnabled(isValid);
    ui.pushButton_media_stop->setEnabled(isValid);
    ui.pushButton_media_repeat->setEnabled(isValid);
}

0

rca 23 jan. 19 at 12:16 am

source to share

Flakker · Accepted Answer · 2015-07-09T12:00:58+0000

So I couldn't get Imem to work, however in the VLC forums I pointed out the new API available in 3.0.0. I had to uninstall the current vlc and libvlc-dev installations and add the daily VLC build to the PPA to my Ubuntu installation and then install those versions.

Libvlc_media_new_callbacks API :

LIBVLC_API libvlc_media_t *     libvlc_media_new_callbacks     (libvlc_instance_t *instance, libvlc_media_open_cb open_cb, libvlc_media_read_cb read_cb, libvlc_media_seek_cb seek_cb, libvlc_media_close_cb close_cb, void *opaque)

You must implement each of these callbacks in order to give VLC access to the in-memory stream. Although the documentation states that the seek () callback implementation is unnecessary, I was unable to get an h264 video without it.

The open () callback should pass a pointer to your video data, I recommend the container class so that you can store the index of the last byte read along with it.

The read () callback is used to read len bytes into the buffer for which the pointer is passed. Here you can write len or fewer bytes to the buffer and return the number of bytes copied, block until you have a few bytes ready, return 0 for EOF or -1 for error.

The seek () callback is used to set the byte index at which the next read () will be performed.

Finally, close () returns nothing and is used to tidy up.

Here's a sample implementation of read ():

class MemVideoData
{
public:
MemVideoData(char *data, int data_bytes) : video(data), bytes(data_bytes), lastByteIndex(0) {}   //init
~MemVideoData() {}
char* video;   //  pointer to video in memory
int bytes;
int lastByteIndex;
};

ssize_t memVideo_read(void *opaque, unsigned char* buf, size_t len)
{
//TODO: block if not end of stream but no bytes available

MemVideoData *mVid = (MemVideoData*) opaque;    //cast and give context
int bytesToCopy=0;
int bytesSoFar = mVid->lastByteIndex;
int bytesRemaining = mVid->bytes - mVid->lastByteIndex;

if(bytesRemaining >= len)   //at least as many bytes remaining as requested
{
    bytesToCopy = len;
}
else if (bytesRemaining < len)    //less that requested number of bytes remaining
{
    bytesToCopy = bytesRemaining;
}
else
{
    return 0;   // no bytes left to copy
}

char *start = mVid->video;
std::memcpy(buf,&start[mVid->lastByteIndex], bytesToCopy);    //copy bytes requested to buffer.
mVid->lastByteIndex = mVid->lastByteIndex + bytesToCopy;    //increment bytes read count

return bytesToCopy;
}

As pointed out here is an example of an open callback:

int VideoPlayer::memVideo_open(void* opaque, void** datap, uint64_t* sizep)
{
   //cast opaque to our video state struct
   MemVideoData *mVid = static_cast<MemVideoData*> (opaque);    

   //TODO: get mutex on MemVideoData object pointed to by opaque
   *sizep = (uint64_t) mVid->bytesTotal;    //set stream length
   *datap = mVid;   /*point to entire object. Think this was intended to 
  point to the raw video data but we use the MemVideoData object in read() 
  and seek()*/

   mVid->lastByteReadIndex=0;

   return 0;
}

Using VLC imem to play h264 video file from memory but getting error "main stream error: cannot prefill buffer"

Edit

More articles: