Wednesday, August 31, 2016

Use Raspberry Pi + HVR1900 to record set-top box output (Telenet digicorder) and stream to DLNA devices

Technically not an OpenSuSE post, I know!
date: Sept. 2016

What you need:
  • Raspberry Pi (I'm using a first generation model B rev 2 with 512MB RAM) and power supply
  • SD card
  • Hauppauge HVR-1900 connected to set-top box (Composite cables to SCART conversion plug)
  • External USB drive

1. Install Raspbian on SD card

Go to and download the Raspbian Jessie Lite (almost everything you need is already included in this image)

Follow the already well documented instructions to write this image to an SD card (and make sure you use a working card)

Make sure the external devices (HDD and HVR-1900) are already connected BEFORE powering on your Pi!

I'm working headless so all I need is the IP address of the Pi after it boots. This page can help you.

Use an SSH-client to connect to the ip address with username "pi" and default password "raspberry".

Now start the built-in configuration assistant to configure basic settings (more info here):

pi@raspberrypi:~ $ sudo raspi-config

┌─────────┤ Raspberry Pi Software Configuration Tool (raspi-config) ├──────────┐
│                                                                              │
│    1 Expand Filesystem              Ensures that all of the SD card s        │
│    2 Change User Password           Change password for the default u        │
│    3 Boot Options                   Choose whether to boot into a des        │
│    4 Wait for Network at Boot       Choose whether to wait for networ        │
│    5 Internationalisation Options   Set up language and regional sett        │
│    6 Enable Camera                  Enable this Pi to work with the R        │
│    7 Add to Rastrack                Add this Pi to the online Raspber        │
│    8 Overclock                      Configure overclocking for your P        │
│    9 Advanced Options               Configure advanced settings              │
│    0 About raspi-config             Information about this configurat        │
│                                                                              │
│                                                                              │
│                     <Select>                     <Finish>                    │
│                                                                              │

  • change the user PASSWORD (option 2)
  • set the correct LOCALE (option 5, sub option 1, nl_BE.UTF-8 UTF-8 in my case)
  • set the correct TIMEZONE (option 3, sub option 2, Europe/Brussels in my case)
Exit the tool and update your system

pi@raspberrypi:~ $ sudo apt-get update
pi@raspberrypi:~ $ sudo apt-get upgrade
pi@raspberrypi:~ $ sudo reboot 

2. Configure HVR-1900

The HVR-1900 is not working out of the box. The drivers and the utilities are already loaded (pvrusb2 module and v4l2-utils are included in the raspbian distribution) but we are still missing the firmware.

Let's check the kernel ring buffer messages for relevant output:

pi@raspberrypi:~ $ dmesg | grep -i pvrusb2
[   11.104384] pvrusb2: Hardware description: WinTV HVR-1900 Model 73xxx
[   11.118840] usbcore: registered new interface driver pvrusb2
[   11.118874] pvrusb2: V4L in-tree version:Hauppauge WinTV-PVR-USB2 MPEG2 Encoder/Tuner
[   11.118890] pvrusb2: Debug mask is 31 (0x1f)
[   12.119585] usb 1-1.3: Direct firmware load for v4l-pvrusb2-73xxx-01.fw failed with error -2
[   12.119629] pvrusb2: ***WARNING*** Device fx2 controller firmware seems to be missing.
[   12.119645] pvrusb2: Did you install the pvrusb2 firmware files in their proper location?
[   12.119660] pvrusb2: request_firmware unable to locate fx2 controller file v4l-pvrusb2-73xxx-01.fw
[   12.119674] pvrusb2: Failure uploading firmware1
[   12.119685] pvrusb2: Device initialization was not successful.
[   12.119696] pvrusb2: Giving up since device microcontroller firmware appears to be missing.
pi@raspberrypi:~ $ 

The HVR-1900 is a device that will load all necessary firmware when it is connected. To get it working you need the correct firmware files in the correct folder and retry the connection.
On some other linux distributions you have to download the appropriate files for your device (or extract them from the official windows drivers) and put them in the correct path (e.g. /lib/firmware)
So download this file and copy it to /lib/firmware

also install some other firmwares that are supplied in packages

pi@raspberrypi:~ $ sudo apt-get install firmware-ivtv
pi@raspberrypi:~ $ sudo apt-get install firmware-linux-nonfree

Now that you have all the necessary files you can reload the driver by typing:

pi@raspberrypi:/lib/firmware $ sudo rmmod pvrusb2
pi@raspberrypi:/lib/firmware $ sudo modprobe pvrusb2 

The dmesg command should now show something like this

pi@raspberrypi:/lib/firmware $ dmesg | tail -n 50
[ 2738.078101] usbcore: deregistering interface driver pvrusb2
[ 2738.078224] pvrusb2: Device being rendered inoperable
[ 2744.193891] pvrusb2: Hardware description: WinTV HVR-1900 Model 73xxx
[ 2744.196793] usbcore: registered new interface driver pvrusb2
[ 2744.196825] pvrusb2: V4L in-tree version:Hauppauge WinTV-PVR-USB2 MPEG2 Encoder/Tuner
[ 2744.196841] pvrusb2: Debug mask is 31 (0x1f)
[ 2745.205158] pvrusb2: Device microcontroller firmware (re)loaded; it should now reset and reconnect.
[ 2745.400786] usb 1-1.3: USB disconnect, device number 5
[ 2745.401383] pvrusb2: Device being rendered inoperable
[ 2747.172979] usb 1-1.3: new high-speed USB device number 6 using dwc_otg
[ 2747.283021] usb 1-1.3: New USB device found, idVendor=2040, idProduct=7300
[ 2747.283059] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2747.283079] usb 1-1.3: Product: WinTV
[ 2747.283096] usb 1-1.3: Manufacturer: Hauppauge
[ 2747.283115] usb 1-1.3: SerialNumber: 7300-00-F077FF16
[ 2747.288188] pvrusb2: Hardware description: WinTV HVR-1900 Model 73xxx
[ 2747.321714] pvrusb2: Binding ir_rx_z8f0811_haup to i2c address 0x71.
[ 2747.321941] pvrusb2: Binding ir_tx_z8f0811_haup to i2c address 0x70.
[ 2747.360449] cx25840 3-0044: cx25843-24 found @ 0x88 (pvrusb2_a)
[ 2747.373770] pvrusb2: Attached sub-driver cx25840
[ 2747.439531] tuner 3-0042: Tuner -1 found with type(s) Radio TV.
[ 2747.439610] pvrusb2: Attached sub-driver tuner
[ 2749.608253] cx25840 3-0044: loaded v4l-cx25840.fw firmware (16382 bytes)
[ 2749.708553] tveeprom 3-00a2: Hauppauge model 73219, rev D2F5, serial# 4034395926
[ 2749.708589] tveeprom 3-00a2: MAC address is 00:0d:fe:77:ff:16
[ 2749.708608] tveeprom 3-00a2: tuner model is NXP 18271C2 (idx 155, type 54)
[ 2749.708627] tveeprom 3-00a2: TV standards PAL(B/G) PAL(I) SECAM(L/L') PAL(D/D1/K) ATSC/DVB Digital (eeprom 0xf4)
[ 2749.708644] tveeprom 3-00a2: audio processor is CX25843 (idx 37)
[ 2749.708659] tveeprom 3-00a2: decoder processor is CX25843 (idx 30)
[ 2749.708675] tveeprom 3-00a2: has radio, has IR receiver, has IR transmitter
[ 2749.708720] pvrusb2: Supported video standard(s) reported available in hardware: PAL-B/B1/D/D1/G/H/I/K;SECAM-B/D/G/H/K/K
[ 2749.708817] pvrusb2: Device initialization completed successfully.
[ 2751.919944] cx25840 3-0044: loaded v4l-cx25840.fw firmware (16382 bytes)
[ 2752.053610] tda829x 3-0042: setting tuner address to 60
[ 2752.105965] tda18271 3-0060: creating new instance
[ 2752.153623] TDA18271HD/C2 detected @ 3-0060
[ 2752.673620] tda18271: performing RF tracking filter calibration
[ 2771.814088] tda18271: RF tracking filter calibration complete
[ 2771.884116] tda829x 3-0042: type set to tda8295+18271
[ 2775.682667] usb 1-1.3: Direct firmware load for v4l-cx2341x-enc.fw failed with error -2
[ 2775.682710] pvrusb2: ***WARNING*** Device encoder firmware seems to be missing.
[ 2775.682725] pvrusb2: Did you install the pvrusb2 firmware files in their proper location?
[ 2775.682739] pvrusb2: request_firmware unable to locate encoder file v4l-cx2341x-enc.fw
[ 2775.683323] pvrusb2: registered device video0 [mpeg]
[ 2775.683361] DVB: registering new adapter (pvrusb2-dvb)
[ 2775.700154] pvrusb2: Clearing driver error statuss
[ 2778.494356] cx25840 3-0044: 0x0000 is not a valid video input!
[ 2778.552382] usb 1-1.3: DVB: registering adapter 0 frontend 0 (NXP TDA10048HN DVB-T)...
[ 2778.563342] tda829x 3-0042: type set to tda8295
[ 2778.602153] tda18271 3-0060: attaching existing instance

Now do a quick test recording

Telenet digicorder specific (and maybe other set-top boxes): The scart output of the digicorder (DC-AD2100) has some quirks. In my case, the digicorder is connected to the TV using HDMI and to the HVR-1900 using scart-to-composite connectors. I found that you need to turn on the TV and watch the content you wish to record otherwise you only get sound. Also, make sure the audio volume of the digicorder is not too high because otherwise you get bad quality.

pi@raspberrypi:~ $ v4l2-ctl --set-input=1 # to activate the composite input
Video input set to 1 (composite: ok)
pi@raspberrypi:~ $ v4l2-ctl -s pal-B # because I live in a PAL-B standard country
Standard set to 00000007
pi@raspberrypi:~ $ cat /dev/video0 > /home/pi/test.mpg
Press CTRL+C after a minute to stop recording... 

Now get the file from your Pi (using an sftp client like filezilla) and check it with an MPEG capable player (like VLC). Since the scart output is interlaced you will need to activate the deinterlace function of you player to get rid of the horizontal lines during quick movement.

3. Automount the external drive

First, get the details of the connected device

pi@raspberrypi:~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="22E0-C711" TYPE="vfat" PARTUUID="dad99d56-01"
/dev/mmcblk0p2: UUID="202638e1-4ce4-45df-9a00-ad725c2537bb" TYPE="ext4" PARTUUID="dad99d56-02"
/dev/sda1: LABEL="tosh1TB" UUID="ce555d59-e22f-4f0b-b8c2-9a77bc2a5374" TYPE="ext4" PARTUUID="148cf869-01"

I'm looking for a Toshiba drive so it is the last line. The UUID is the unique number of this drive that well remain the same across reboot or even on other machines.

Now we need to mount the device to a local folder for easy access. To do this you need to create a subfolder at /mnt.

pi@raspberrypi:~ $ sudo mkdir /mnt/usb
pi@raspberrypi:~ $ sudo chown -R pi:pi /mnt/usb #to get correct permissions on that folder

Now edit the /etc/fstab file and add one line to the end. Press CTRL+X to exit and save the file:

pi@raspberrypi:~ $ sudo nano /etc/fstab

  GNU nano 2.2.6               File: /etc/fstab                                    

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
UUID="ce555d59-e22f-4f0b-b8c2-9a77bc2a5374" /mnt/usb ext4 auto,users 0 0

^G Get Help  ^O WriteOut  ^R Read File ^Y Prev Page ^K Cut Text  ^C Cur Pos
^X Exit      ^J Justify   ^W Where Is  ^V Next Page ^U UnCut Text^T To Spell

UUID is the unique ID for that disk
/mnt/usb is the mountpoint you want to use
ext4 is the filesystem type for this disk
auto,users are settings related for automount and security
0 0 is for dump and checkdisk settings

if you now reboot or issue this command...

pi@raspberrypi:~ $ sudo mount -a
pi@raspberrypi:~ $ ls -ltr /mnt/usb
total 28
drwx------  2 pi users 16384 Jul 24  2013 lost+found
drwxr-xr-x  2 pi users  4096 Oct 20  2013 tmp
drwxr-xr-x 28 pi users  4096 Dec 30  2015 Maxtor
drwxr-xr-x  2 pi users  4096 Aug 31 19:23 capture
pi@raspberrypi:~ $ 

you can see that the contents of the disk are available at /mnt/usb.
I created a capture folder to keep all captures.

4. Remote control your setup (start/stop)

Our goal is to use a simple web interface to start and stop the recording so you do not need to use a SSH for this.
To generate the web interface we will use the Flask framework because it's simple and also has a web server built-in.
Install it (and all dependencies) with:

pi@raspberrypi:~ $ sudo apt-get install python3-flask

It can take a while, afterward we need to create 2 files: the main python program and the html template.

NOTE: You will have to adapt this program because every HVR-1900 device has its own serial number. Only one line to adjust:
/sys/class/pvrusb2/sn-4034395926 <== change this number

pi@raspberrypi:~ $ nano /home/pi/
now paste this text
import subprocess
import datetime
import time
from flask import Flask
from flask import render_template

app = Flask(__name__) 
@app.route('/pvr2web/', defaults={'action': None}, methods=['POST','GET'])
@app.route('/pvr2web/<action>', methods=['POST','GET'])
def process_action(action):
    if action == 'start':
        start='{:%Y-%m-%d_%H:%M:%S}'.format(["v4l2-ctl","--set-input=1"])["v4l2-ctl","-s","pal-B"])"cat /dev/video0 > /mnt/usb/capture/" + start +".mpg &",shell=True)
    elif action =='stop':"killall cat",shell=True)
    stdoutdata = subprocess.getoutput("cat /sys/class/pvrusb2/sn-4034395926/ctl_streaming_enabled/cur_val")
    if stdoutdata == 'true':
        action = 'Recording'
        action = None
    return render_template('index2.html',action=action)
if __name__ == '__main__':'',debug=True)  
Press CTRL+X to exit and save, then run 
 pi@raspberrypi:~ $ chmod +x #to make it executable

For the html template we need to create a subfolder

pi@raspberrypi:~ $ mkdir /home/pi/templates
pi@raspberrypi:~ $ nano /home/pi/templates/index2.html
paste the following text 

<!DOCTYPE html>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
h1 {
    text-align: center;
    background-color: lightgray;
    padding: 25px 25px;

h2 {
    color: lightgray;
    padding: 25px 25px;

.button {
    background-color: #4CAF50;
    border: none;
    color: white;
    padding: 15px 32px;
    text-align: center;
    text-decoration: none;
    display: inline-block;
    font-size: 24px;
    margin: 4px 4px;
    cursor: pointer;

.button2 {background-color: #f44336;} /* Red */ 

<h1>Raspberry PI PVR</h1>
<div style="text-align:center;">
{% if action %}
<a href="/pvr2web/stop" class="button button2">Stop Recording</a>
{% else %}
<a href="/pvr2web/start" class="button">Start Recording</a>
{% endif %}

Press CTRL+X and save

Start the program to give it a try:

pi@raspberrypi:~ $ /home/pi/ 
 * Running on
 * Restarting with reloader

Now go to a web browser and try it out: http://ip-address-of-pi:5000/pvr2web
You should see the following:

Now press the green button...

Press stop... This is what you should see in the console output

pi@raspberrypi:~ $ /home/pi/ 
 * Running on
 * Restarting with reloader
Video input set to 1 (composite: ok)
Standard set to 00000007 - - [31/Aug/2016 19:53:30] "GET /pvr2web/start HTTP/1.1" 200 - - - [31/Aug/2016 19:54:10] "GET /pvr2web/stop HTTP/1.1" 200 -

A file with a timestamp should now exist in the target path

pi@raspberrypi:~ $ ls -ltr /mnt/usb/capture/
total 30146560
-rw-r--r-- 1 pi pi   30146560 Aug 31 19:54 2016-08-31_19:53:29.mpg
pi@raspberrypi:~ $ 

now we only have to autostart this program at boot time. We are going to use CRON for this because it is already installed. It just requires adding one line to a file.

pi@raspberrypi:~ $ crontab -e
no crontab for pi - using an empty one

/usr/bin/select-editor: 1: /usr/bin/select-editor: gettext: not found
  1. /bin/ed
/usr/bin/select-editor: 1: /usr/bin/select-editor: gettext: not found
  2. /bin/nano        <---- 
  3. /usr/bin/vim.tiny

/usr/bin/select-editor: 32: /usr/bin/select-editor: gettext: not found
 1-3 [2]: 2

and add this line at the bottom: @reboot /home/pi/ &

exit and save using CTRL+X and reboot...

5. Install minidlna (ReadyMedia) to stream to DLNA clients

install minidlna and change the library path

pi@raspberrypi:~ $ sudo apt-get install minidlna
pi@raspberrypi:~ $ sudo nano /etc/minidlna.conf 

change "media_dir=/var/lib/minidlna"
to "media_dir=/mnt/usb/capture"
Press CTRL + X to exit and save 

Now restart the service

pi@raspberrypi:~ $ sudo systemctl restart minidlna

And try to connect using VLC


Optional: use NFS to connect from *NIX clients.

NFS gives you the highest transfer between the pi and others.

Installation is easy:

pi@raspberrypi:~ $ sudo apt-get install nfs-common nfs-kernel-server

configuration is done by adding 2 lines to a config file:

pi@raspberrypi:~ $ sudo nano /etc/exports

just add the 2 last lines and press CTRL+X to exit and save

  GNU nano 2.2.6             File: /etc/exports                       Modified  

# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_sub$
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)

^G Get Help  ^O WriteOut  ^R Read File ^Y Prev Page ^K Cut Text  ^C Cur Pos
^X Exit      ^J Justify   ^W Where Is  ^V Next Page ^U UnCut Text^T To Spell

Now reload NFS by issuing:

pi@raspberrypi:~ $ sudo exportfs -rav

pi@raspberrypi:/mnt/usb/capture $ sudo systemctl enable rpcbind
pi@raspberrypi:/mnt/usb/capture $ sudo systemctl start rpcbind
pi@raspberrypi:/mnt/usb/capture $ sudo systemctl restart nfs-kernel-server

From another machine you can now connect using...

sudo mkdir /mnt/nfs
sudo mount /mnt/nfs