tag:blogger.com,1999:blog-82359985700697825862024-03-13T08:08:43.643+09:00rikijpn's blogrikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.comBlogger125125tag:blogger.com,1999:blog-8235998570069782586.post-4246499474437317812022-03-18T18:48:00.002+09:002022-03-18T18:48:45.690+09:00how to shrink disk on qcow2 image (with an xfs filesystem)<h2>abstract</h2>
This explains how to shrink (or change the size of the disk) of a qcow2 image.<div>qcow2 images are used in VMware virtual machines.<br />I am converting everything to vdi to test with virtualbox, so I can test directly in my computer.<br /><br /></div><div>Basically, you can't directly shrink a qcow2 image (using xfs). The actual process is to make another image (with a different disk size), copying all the existing contents from the original one.</div><div>For simplicity, this image is a GNU/Linux system (centos), with a grub2 bootloader.</div><div>I'm doing all these commands on a Debian 11 system.
<h2>steps overall</h2>
1. convert qcow2 to vdi</div><div>2. create new disk for new disk size, and another to locally save the backup (temp)</div><div>3. attach the new and the old image disks (vdi) to the VM</div><div>4. create new filesystems, mount and backup/restore the old image files</div><div>5. copy mbr to new system</div><div>6. update `/boot/grub2` and `/etc/fstab` for the new disk's uuid, and remove grub2 admin password (in case your original image had the grub2 admin password set)</div><div>7. convert vdi to qcow2
<h2>pre-requirements</h2>
1. the qcow2 image (can download from openstack)</div><div>2. virtualbox with a working GNU/Linux OS (to run from a live CD ISO image is OK too)
<h2>steps with commands in detail</h2>
<pre><code>
# 1. convert qcow2 to vdi
# (get the image to your local machine first)
qemu-img convert -O vdi some_image.qcow2 some_image_backup.vdi
# 2. create new disk for new disk size, and another to locally save the backup (temp)
# use virtualbox's GUI for this
# create new_disk.vdi. And temp_disk.vdi if you are running a live CD. Using your local disk is OK too.
# 3. attach the new and the old image image disks (vdi) to the VM
# use virtualbox's GUI for this. It's done in the "storage" tab
# 4. create new filesystems, mount and backup/restore the old image files
cd /mnt
mkdir old_random_image
mkdir new_random_image
mkdir temp_disk
fdisk -l /dev/sda
n
ENTER bunch of times
w
# same for /dev/sdb
mkfs.ext4 /dev/sda1
mkfs.xfs /dev/sdb1
mount /dev/sda1 temp_disk
mount /dev/sdb1 new_random_image
mount /dev/sdc1 old_random_image
cd temp_disk
apt-get update
apt-get install xfsdump parted
xfsdump -f random_image.dump /mnt/old_random_image
# use random names for job and disk labels like: foo, foo2, bar, etc
xfsrestore -f random_image.dump /mnt/new_random_image
# use df -h and ls to confirm the filesystems seem to match
# add boot permission
# check old image
parted /dev/sdc
p
Ctrl-D
# add boot flag to partition
parted /dev/sdb
p
toggle 1
boot
p
Ctrl-D
# 5. copy mbr to new system
# backup old mbr
dd if=/dev/sdc of=orig_mbr bs=512 count=1
# copy MBR to new disk (booting part only, without the partition table definition)
dd if=orig_mbr of=/dev/sdb bs=446 count=1
# 6. update `/boot/grub2` and `/etc/fstab` for the new disk's uuid, and remove grub2 admin password
cp -r /boot/grub2 /boot/grub2_bk
cd /boot/grub2
rm user.cfg grub.cfg.*.rpmsave
# check the new disk's uuid
ls -lh /dev/disk/by-uuid/
# replace the old uuid with the new one, and remove password requiring config
vi grub.cfg
################################################################################
# (here is a diff for above)
78,87d80
< ### BEGIN /etc/grub.d/01_users ###
< if [ -f ${prefix}/user.cfg ]; then
< source ${prefix}/user.cfg
< if [ -n "${GRUB2_PASSWORD}" ]; then
< set superusers="root"
< export superusers
< password_pbkdf2 root ${GRUB2_PASSWORD}
< fi
< fi
< ### END /etc/grub.d/01_users ###
90c83
< menuentry 'CentOS Linux' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-XXXX-OLD-DISK-UID' {
---
> menuentry 'CentOS Linux' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-XXXX-NEW-DISK-UID' {
98c91
< search --no-floppy --fs-uuid --set=root --hint='hd0,msdos1' OLD-DISK-UID
---
> search --no-floppy --fs-uuid --set=root --hint='hd0,msdos1' NEW-DISK-UID
100c93
< search --no-floppy --fs-uuid --set=root OLD-DISK-UID
---
> search --no-floppy --fs-uuid --set=root NEW-DISK-UID
102c95
< linux16 /boot/vmlinuz-XXX.x86_64 root=UUID=OLD-DISK-UID ro console=tty0 crashkernel=auto console=ttyS0,115200 audit=1 LANG=en_US.UTF-8
---
> linux16 /boot/vmlinuz-XXX.x86_64 root=UUID=NEW-DISK-UID ro console=tty0 crashkernel=auto console=ttyS0,115200 audit=1 LANG=en_US.UTF-8
105c98
# ...
################################################################################
# do the same for fstab
vi /etc/fstab
# should look like this:
# UUID=XXXXXXXXXXXXXXXXXXX / xfs defaults 0 0
# (this has the new disk's UUID)
# restart VM unmounting all disks but the new image
# and check it boots OK (probably can't get a prompt due to network settings, but you should
# get no kernel error, or booting errors like "no disk/booter found" etc)
# 7. convert vdi to qcow2
qemu-img convert -f vdi -O qcow2 ~/virtualbox_vms/disks/random_image10G.vdi some_image_10G.qcow2
# 8. upload image to openstack</code>
openstack image create --disk-format qcow2 --container-format bare --file image/some_image_10G.qcow2 img-name-in-openstack</pre><pre></pre>
</div>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-3229968774290216692022-03-18T18:28:00.003+09:002022-03-18T18:28:37.112+09:00booting cisco iosxrv in virtualbox<h2 style="text-align: left;">abstract</h2><div>Cisco has some virtual images of their iosxr system (iosxrv), which aren't very easy to use.</div><div>This is how I got iosxrv to work on virtualbox.</div><div>The trickiest parts were that the specs fixed/imported by image are pretty large, so you have to scale them down to make it work. And that virtualbox apparently can't find the interface to do ssh forwarding so you can actually connect to the device (is my GUI client buggy?... idk, but found it in the xml file you're not supposed to edit manually, but that works if you do anyway).</div><p><br /></p><h2 style="text-align: left;">overall steps</h2><p>1. import ova image</p><p>2. change image specs/configs</p><p>3. boot image </p><p>4. (on X) create default user</p><p>5. (on X) setup ssh server and interface</p><p>6. confirm you can ssh with a localhost high port</p><p><br /></p><h2 style="text-align: left;">Prerequisites</h2><p>1. virtualbox</p><p>2. image files (ova files)</p><p>3. an X working system (connect from a GNU/Linux system on your client end, or install a windows client)</p><p>I did this on my debian server, on both a physical machine, and a VM, worked just fine.</p><p><br /></p><h2 style="text-align: left;">steps</h2><p><br /></p><code><p># ssh to the VM with the -X option, so you can run X programs from the server<br />ssh -X YOUR_SERVERS_IP_ADDR</p><p><br /></p><p># sample for 6.5.2. Please replace VM/image name as needed<br /># import image</p><p>VBoxManage import xrv9k-fullk9-x.vrr.vga-6.5.2.ova</p><p><br /></p><p># change name<br />VBoxManage modifyvm com.cisco.ios-xrv9000 --name iosxrv6.5.2</p><p><br /></p><p># change default options that make VM unbootable<br />VBoxManage modifyvm iosxrv6.5.2 --vtxux on<br />VBoxManage modifyvm iosxrv6.5.2 --ostype Linux_64<br />VBoxManage modifyvm iosxrv6.5.2 --cpus 6<br />VBoxManage modifyvm iosxrv6.5.2 --memory 7096<br />VBoxManage modifyvm iosxrv6.5.2 --vram 10<br />VBoxManage modifyvm iosxrv6.5.2 --longmode on<br />VBoxManage modifyvm iosxrv6.5.2 --graphicscontroller vmsvga</p><p><br /></p><p># add ssh port forwarding settings from virtualbox<br />VBoxManage modifyvm iosxrv6.5.2 --nic6 nat</p><p># Change local port ("5801" here) and connection name (ssh_652) per VM.<br /># The rest stays the same<br />VBoxManage modifyvm iosxrv6.5.2 --natpf6 ssh_652,tcp,127.0.0.1,5801,10.0.7.15,22</p><p><br /></p><p># check mac address<br />grep "Adapter slot=\"5\"" ~/ ~/VirtualBox\ VMs/iosxr\ 6.5.2/*box # replace dir as necessary</p><p># <Adapter slot="5" enabled="true" MACAddress="XXXX" cable="true" type="XXXX"><br /># we'll use this to confirm on the VM side later that the right interface<br /># is being addressed</p><p># boot once with X on<br /># it will be super slow (like 10m?), but it should boot. And ask you for your<br /># new root username/password</p><p>VBoxManage startvm iosxrv6.5.2</p><p><br /></p><p>##################<br /># on the X window<br />##################</p><p># create the new user following the prompt's instructions</p><p><br /></p><p># name the device, and setup the ssh server<br />conf t<br />hostname iosxr652<br />commit<br />exit</p><p>crypto key generate rsa<br />conf t<br />ssh server v2<br />line default transport input ssh<br />commit<br />exit</p><p># setup interface for ssh forwarding<br />show int MgmtEth0/RP0/CPU0/0</p><p># confirm mac address is the right one for our port forward<br /># you should get somewhere around the top a string representing the mac address<br /># check it matches the string got in the above step "check mac address"<br /># if not, to check which MAC address matches the mgmt interface, and do the NAT<br /># ssh port settings to that one instead</p><p>conf t<br />interface MgmtEth0/RP0/CPU0/0<br /> ipv4 address dhcp<br /> no shutdown<br />commit<br />exit<br />exit</p><p><br /></p><p># confirm interface is up and has an IP address assigned<br />show int MgmtEth0/RP0/CPU0/0</p><p># we're assuming the IP address is 10.0.7.15<br /># if not, change NAT ssh forwarding settings accordingly</p><p>########################<br /># outside the X window<br /># (on your regular terminal)<br />########################</p><p># (on the host running virtualbox)<br />ssh random_admin@localhost -p 5801 # replace username as necessary<br /><br /></p><p># you should be able to login<br /># click the close button on the X window with the VM, and choose "shutdown VM"</p><p># start VM again, with no X window<br />VBoxManage startvm --type headless iosxrv6.5.2<br /><br /></p><p># after 10m or so, confirm you can ssh again<br />ssh random_admin@localhost -p 5801</p><p># that's it!</p><p># to stop:<br /># VBoxManage controlvm iosxrv6.5.2 poweroff </p><p><br /></p></code>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-68439906929453786552020-11-16T16:36:00.002+09:002020-11-16T16:36:30.246+09:00moving lots of files with BOX<p>I had to move like 300K files in BOX, so just sharing a bit of how I did it and how to check the progress.</p><p>Basically, get:</p><p>1. BOX drive (no gnu/linux version available, so mounting the "folder" on a gnu/linux VM as a shared dir, and working from there)</p><p>2. A gnu/linux VM (I use virtualbox) with the necessary kernel stuff to mount shared dirs (thing to mount vboxfs)</p><p><br /></p><p>I guess you could have done something similar in windows or mac too just writing a program directly in those platforms. But I hate both OSs, so, no.</p><p><br /></p><p>You just write the program to rename/move the files however you want, and run it as with any other dir.</p><p><br /></p><p>The catchy part is that box is super slow (for me at least), so even when in your disk all files would have been updated fairly fast, in the background, the thing takes forever to finish the sync... so if you check in the web version, many of the files would still be in the old location/will have their old filenames.</p><p>There should be a file called Box-{version?}.log in:</p><p>c:/Users/$USER/AppData/Local/Box/Box/logs/</p><p><br /></p><p>There you should be able to see logs in the format:</p><p>DATE TIME ID? INFO LocalExecutor-2 box_fs_sync_api Move item on box. XXXX</p><p>Telling you what it's doing.<br />I also mounted this in my VM and checked the logs with tail -f.</p><p><br /></p><p>As of the time of this post, there is apparently no way to tell the sync progress besides this... And you basically will have to grep the log and compare with the list of files done with whatever you have in order to have an idea of the total percentage.</p><p>Not the coolest system.</p><p>Until the sync is complete, your box drive icon will show "Box Drive is updating your files" on mouse over. And it takes really super long... But maybe it's because of the many requests I sent, so my account was limited or something. It's like 1~3 requests a second based on the logs above.</p><p>If you're luckier than me, it should take under 3 days. If not, I hope you have a quiet computer.</p>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-68507979706149084392020-05-14T20:17:00.000+09:002020-05-14T20:17:26.617+09:00installing gnu/linux in DELL G5 5090So, first time ever not buying a used computer or server. Wanting to enjoy 4K and everything, I bought Dell's G5 5090, obviously to use with GNU/Linux.<div><br /></div><h1 style="text-align: left;">First impressions</h1><div>1. the blue thingy is super annoying. Why is it so damn bright... I guess it must be a gaming/visual thing that makes no sense for engineers. (I was able to disable the LED from the bios later)</div><div>2. besides that, it looks awesome, lots of USB slots, a nice video card, HDMI interface, and lots of PCI slots to add more stuff if I need to.</div><div><br /></div><h1 style="text-align: left;">After opening it</h1><div>1. apparently it only has 2 HDDs slots... more would be nicer. I'll just use my external stands I guess.</div><div>2. holy sh*t the NVME disk is tiny</div><div><br /></div><h1 style="text-align: left;">Installing GNU/linux</h1><div><ol style="text-align: left;"><li>The first issue is that by default, it won't let you access the BIOS... no matter how many times you press the F2 button. It immediately boots windows, and gets you that incredibly ugly and hard to use "compulsory" setup screen.<br />For god's sake, windows is annoying.<br />So, we have reset the CMOS so we can enter the BIOS setup screen, and choose another disk to boot (and disable the UEFI "secure" boot thing). Let's do that...</li><li>Resetting the CMOS is pretty easy actually. You just open the cover, and move the jumper from the PASSWD slot to the CMOS_CLEAR or something slot. Here is a reference document: <a href="https://www.dell.com/support/article/ja-jp/sln284985/how-to-perform-a-bios-or-cmos-reset-and-clear-the-nvram-on-dell-systems?lang=en#Shortcut_2">https://www.dell.com/support/article/ja-jp/sln284985/how-to-perform-a-bios-or-cmos-reset-and-clear-the-nvram-on-dell-systems?lang=en#Shortcut_2</a> </li><li>now we can see the bios screen, yay. You will need to:</li><ol><li>disable UEFI secure boot (so you access your GNU/Linux install disk prob)</li><li>change the SATA controller from intel RAID to AHCI (so you can actually use your NVME disk from linux)</li></ol><li>that's basically it. Now you can just insert your USB stick with the GNU/Linux install (I recommend Debian 10, whatever you choose, just no f*cking ubuntu ok?), and it should boot on it by default. If that doesn't work for you, access the BIOS again, go to the boot order screen, and make sure your linux disk is at the top.<br />Partition notes: If you want to boot from your NVME disk, remember to create an EFI on the partition guide. Otherwise you'll have to boot from an external disk just for the boot loader.<br /><br /></li></ol><h3 style="text-align: left;">Warning</h3></div><div>One more really annoying thing, is that apparently windows rewrites the bios settings once you boot it. So if you by mistake boot in windows, the next time you start your computer you won't be able to access the BIOS again. So you'll have to do the CMOS reset once again... really really really get rid of that piece of sh*t OS.</div><div><br /></div><h1 style="text-align: left;">Message for Dell</h1><div>Please sell it to me cheaper, and without windows. That's a real win-win situation.</div><div>I also didn't need the mouse and keyboard... which is not even in a keymap I can/like to use.</div><div><br /></div>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-29517856100207536352020-04-30T18:45:00.004+09:002020-04-30T18:45:54.921+09:00workaround for MySQL server has gone away error with flaskI had this issue when my flask simple app kept failing the next day it was started.<br />It kept giving the error "MySQL server has gone away".<br />
<br />
I tried a couple of things like, starting the mysql session for every request instead of for each flask process (app-wide). But apparently flask doesn't like that.<br />
At least not when using flaskext.mysql's MySQL.<br />
<br />
I think I didn't have much luck trying to change the timeout on flask either, as this module just didn't let me.<br />
<br />
I could have changed the module to use mysql, but I didn't have much time to finish the damn thing, so in the end I just set a cron job to access the mysql select query using page every 10 min or so. Ugly, but simple workaround.<br />
<br />
It was like<br />
curl -k https://localhost/some_app/index.html<br />
<br />
every 10m.<br />
<br />rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-17034930907731582032020-04-30T18:20:00.000+09:002020-04-30T18:20:22.932+09:00flask app returns select with old dataI had this issue where my flask app kept giving me different results (doing a select query).<br />
<br />
And it only happened on production, not in the development one.<br />
Apparently due to the fact that flask has a couple of fork processes running, each with their own mysql session.<br />
Even when you commit after an insert/update query, you still need for some reason to do a commit before a select in order to see the latest results.<br />
<br />
The following is a sample code with some basic functions in order to do select and update queries.<br />
I just call validate_sql_conn() for pretty much everything I need to do in mysql, so it's always committing itself.<br />
<br />
<br />
<pre>from flask import Flask, render_template, url_for, request, send_from_directory
from flaskext.mysql import MySQL
import yaml
app = Flask(__name__, template_folder='views')
def get_mysql_conf(mysql_conf_file="some_app_name_conf/mysql_conf.yml"):
"gets the conf to use for the mysql connection"
with open(mysql_conf_file, 'r') as f:
content = f.read()
yaml_content = yaml.load(content, Loader=yaml.FullLoader)
return yaml_content
mysql = MySQL()
mysql_conf = get_mysql_conf()
app.config['MYSQL_DATABASE_USER'] = mysql_conf['mysql_database_user']
app.config['MYSQL_DATABASE_PASSWORD'] = mysql_conf['mysql_database_password']
app.config['MYSQL_DATABASE_DB'] = mysql_conf['mysql_database_db']
app.config['MYSQL_DATABASE_HOST'] = mysql_conf['mysql_database_host']
mysql.init_app(app)
conn = mysql.connect()
cursor = conn.cursor()
def validate_sql_conn():
"""
uses global connection variable (conn) and recreates
it if seems to have dropped
"""
global conn
global cursor
try:
cursor.execute("show tables")
except:
conn = mysql.connect()
cursor = conn.cursor()
conn.commit()
return True
def run_sql_and_get_results(sql_query, cursor=cursor):
validate_sql_conn()
cursor.execute(sql_query)
data = cursor.fetchall()
return data
def run_sql_and_commit(sql_query, cursor=cursor, conn=conn):
validate_sql_conn()
cursor.execute(sql_query)
data = cursor.fetchall()
commit_data = conn.commit()
return commit_data
</pre>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-61080936553784763422020-04-16T16:10:00.001+09:002020-04-16T16:10:34.875+09:00gnus filtering html body mailIf you love emacs, you probably use gnus, which is in my opinion the
<br />
best mailer ever.
<br />
You can manage your complete filter in emacs lisp, and there are so many
<br />
ways to filter stuff.
<br />
<br />
The most complex and useful filtering method is using
<br />
"nnmail-split-fancy".
<br />
<br />
Which as I'm showing in this example, can be set as something like this:
<br />
<br />
<pre>(setq nnmail-split-fancy
'(|
("subject" ".*/var/log/messages.*" "server_syslogs")
(any "some_sender" "some_annoying_sender")
;; to me, not mailing lists
(any ".*MY_NAME.*"
(|
(from ".*some_from_field.*" "some_company")
(from "some_mailer_that_needs_body_filtering"
(|
(: split-on-body ".*Assignee: MY_NAME.*" "your_service_me_assigned")
(: split-on-body ".*Reporter: MY_NAME.*" "your_service_me_requested")
"your_crappy_service")))
"me_somewhere"
))
"misc")
</pre>
<br />
This would split stuff by subject first (having /var/log/messages in the
subject), then see if the header contains "some_sender" and put the mail
in the mail folder "some_annoying_sender". If neither of those matched,
then would look for MY_NAME in the mail header, and split the logic even
further, to split from the "from" header, and even on the mail body. It
can be any function, but this "split-on-body" one is a common one I
guess, since it comes with the documentation and all.
<br />
This is the split-on-body function:
<br />
<pre>(defun split-on-body (regexp-to-search group-to-split)
(save-excursion
(save-restriction
(widen)
(goto-char (point-min))
(when (re-search-forward regexp-to-search nil t)
group-to-split))))
</pre>
<br />
And it works great. But you get issues when filtering HTML mail... which
gnus kind of usually parses someway (when you read it on gnus, it'll
depend on the value of the "mm-text-html-renderer" variable).
<br />
The thing is, when the mail is being filtered, the text apparently is
all in pure text, and that text kind of differs from the actual raw mail
body, or what you see parsed in the mail view.
<br />
So, in short, you'll need to get the correct regexp, which you can't
see anywhere but in the filter function itself...
<br />
So, here is a filter function for that purpose. It's pretty much the
same as the above, but you get the prin1 text for the complete mail text
in you *Messages* buffer.
<br />
<pre>;; debug version
(defun split-on-body (regexp-to-search group-to-split)
"debug version"
(save-excursion
(save-restriction
(widen)
(goto-char (point-min))
(message "searching for regexp in split on body")
(let ((search-results (re-search-forward regexp-to-search nil t))
(complete-buffer (prin1 (buffer-substring (point-min) (point-max))))
)
(message (concat "regexp to search for = " regexp-to-search))
(message (concat "search results =" (format "%S" search-results))))
(goto-char (point-min))
(when (re-search-forward regexp-to-search nil t)
group-to-split))))
</pre>
<br />
For some reason I didn't think of this until like 10 years after
<br />
starting using gnus...rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-51703894784024045022020-04-13T17:06:00.000+09:002020-04-13T17:06:29.999+09:00rsyslog change output log file based on hostnameI want to put all the logs for hostnames starting with XXX in
<br />
/var/log/XXX.log.
<br />
<br />
And I'd also like to be able to read the file without having to do sudo
<br />
everytime...
<br />
<br />
Here is how:
<br />
<br />
<h2>
in /etc/rsyslog.conf:</h2>
<pre># remember to open your udp port to receive logs from other servers
$ModLoad imudp
$UDPServerRun 514
#### GLOBAL DIRECTIVES ####
# change umask so the default one doesn't mess with your filecreatemode permissions
$umask 0000
</pre>
<pre>
</pre>
<h2>
then create a file in /etc/rsyslog.d/50-my-XX-logs.conf:</h2>
<pre>#(any number is fine)</pre>
<pre>
</pre>
<pre># this makes the file readable by anyone
$FileCreateMode 0644
:HOSTNAME, startswith, "XXX" /var/log/UHN2.log
# and stop any further filtering with the next line
& stop
</pre>
This made it work for me. You can check more on the filters and
<br />
conditionals with rsyslog.conf(5) manual page.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-14109590725214570502020-04-11T11:51:00.002+09:002020-04-11T11:52:08.133+09:00use youtube-dl with sites that need credentials/cookiesYou might want to download something with youtube-dl, and get the following error:
<br />
<br />
ERROR: XXXX requires authentication. You may want to use --cookies.
<br />
<br />
<br />
Quick way to get the cookies and do the download:
<br />
1. get the chromium/chrome extension Editthiscookie
<br />
<a href="https://chrome.google.com/webstore/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg/related?hl=en">https://chrome.google.com/webstore/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg/related?hl=en</a>
<br />
<br />
2. on editthiscookie's settings, go to options -> "Choose the preferred
<br />
export format for cookies" -> "Netscape HTTP Cookie File"
<br />
<br />
3. on the site you're trying to download the video, click on
<br />
editthiscookie's extension, export
<br />
<br />
4. paste/yank your clipboard's contents into a file
<br />
<br />
5. do youtube-dl --cookies exported_cookies_file <a href="https://the-vid-you-wanted-to-download/">https://the-vid-you-wanted-to-download</a>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-42430232972471629652020-04-11T10:00:00.000+09:002020-04-11T10:00:33.646+09:00fix emacs package-list's "Failed to verify signature..."Everytime I called M-x package-list-packages, I got
<br />
<br />
package--check-signature: Failed to verify signature
<br />
some_package.el.sig: ("No public key for XXX")
<br />
<br />
I'm not really sure what effect this had, but it was annoying.
<br />
<br />
<br />
Fix
<br />
<br />
1. in a random buffer, do
<br />
<br />
(setq package-check-signature nil) ;; press C-x C-e here
<br />
<br />
2. do M-x package-list-packages
<br />
search for gnu-elpa (complete name: gnu-elpa-keyring-update)
<br />
and install that
<br />
<br />
3. then just restart emacs or do
<br />
(setq package-check-signature 'allow-unsigned) ;; press C-x C-e here
<br />
<br />
<br />
The next time you run package-list-packages, the warning should be gone.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-80065636345439547312020-04-11T09:46:00.001+09:002020-04-11T09:46:31.710+09:00emacs as rest client (API testing interface, like postman)I love being able to use emacs as a REST API testing interface, kinda like postman, but in emacs.
<br />
<br />
Here are two ways to do this:
<br />
<br />
- restclient-mode <a href="http://emacsrocks.com/e15.html">http://emacsrocks.com/e15.html</a>
<br />
- walkman <a href="https://github.com/abrochard/walkman">https://github.com/abrochard/walkman</a>
<br />
<br />
<h2>
In short</h2>
Restclient should be your thing 90% of the time.
<br />
walkman prob 10%. But walkman works inside org-mode, so if you like org,
<br />
maybe can be 100% your thing.<br />
<br />
<h2>
In long</h2>
<br />
Restclient's thing even has a cute video and everything. So,
<br />
noob-friendly.
<br />
Resclient is probably the de facto standard right now to use in this
<br />
kind of thing though.
<br />
It's awesome, you can easily use variables, which can be plain elisp,
<br />
and it uses emacs' internal http request libraries for HTTP interaction.
<br />
The output is json-prettified and all, and requests are done async, very
<br />
nice.
<br />
<br />
The only thing, is that those libraries appear to be a bit buggy... and
<br />
sometimes won't work as expected.
<br />
I have this thing, that for this specific site, I always get a 404 or
<br />
500 status code.
<br />
But when I generate the curl request from the exact same block, it works
<br />
(curl works, elisp's request library apparently doesn't like my
<br />
headers ?).
<br />
<br />
So for that API, I use walkman inside org-mode, which uses pure curl,
<br />
and that works perfectly.
<br />
<br />
I like walkman a lot, as I love org-mode.
<br />
But the variable definitions must done in elisp, and you have to quote
<br />
them to use them... which is not very readable (and elisp code doesn't
<br />
look too good in org mode).
<br />
<br />
<h2>
Samples</h2>
<h3>
restclient</h3>
<pre># -*- mode: restclient; -*-
# block1 gets fsf's site
GET https://www.fsf.org
# press C-c C-c somewhere in this block to send the request
# block2 POSTing stuff with a payload
POST https://your-fav-api/v1/resource
YOUR-HEADERS-HERE
{your payload
can use multiple lines
}
# block3 ...
</pre>
<br />
<h3>
walkman
</h3>
<br />
<pre>* get a page
GET https://www.fsf.org
# you do C-c C-RET here to send the request
* post something
POST https://your-favorite-api/v1/resource
- HEADER1
- HEADER2
{ PAYLOAD-HERE }
</pre>
<br />
That's about it. They're both pretty intuitive, and the documentation is
<br />
clear enough to let you do whatever you want within like 5mins of
<br />
looking for it.
<br />
<br />
I'd usually want to test/use the API interface in a different file. So I
<br />
guess I'd be using restclient mainly, linking it from my "task" file in
<br />
org-mode. Mostly because the variables, and output are more readable
<br />
(highlighted and everything).
<br />
And when that fails, use walkman.
<br />
<br />
But that's just me.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-16926842080372974392020-04-02T16:29:00.000+09:002020-04-02T16:29:31.854+09:00playing dvds in debian10Steps<br />
1. add the contrib (and probably non-free) repo to your sources list<br />
2. install the libdvd-pkg package, and a player<br />
3. run the libdvdcss library installer<br />
4. (might be necessary to) set the region for your dvd device<br />
5. try playing it<br />
<br />
From top to bottom<br />
if necessary, change your sources list<br />
<br />
$ cat /etc/apt/sources.list<br />
deb http://debian-mirror.sakura.ne.jp/debian/ buster main non-free contrib<br />
deb-src http://debian-mirror.sakura.ne.jp/debian/ buster main non-free contrib<br />
<div>
...</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
(you need to have the contrib repo as above)</div>
<div>
if you did any changes, do "sudo apt-get update" to fetch the new repo data.</div>
<div>
<br /></div>
<div>
install the libdvd-pkg</div>
<div>
<br /></div>
<div>
$ sudo apt-get install libdvd-pkg</div>
<div>
or better</div>
<div>
$ sudo apt-get install libdvd-pkg libdvdcss-dev libdvdcss2 libdvdcss2-dbgsym libdvdnav4 libdvdread4 lsdvd regionset mpv lsdvd</div>
<div>
<br /></div>
<div>
then run the configure script, to compile the libdvdcss library</div>
<div>
$ sudo dpkg-reconfigure libdvd-pkg</div>
<div>
<br /></div>
<div>
I don't remember if you needed to do this the first time, or just if your dvd wouldn't play... but try inserting a dvd, and do</div>
<div>
$ sudo regionset</div>
<div>
Some commercial DVDs won't play unless you have the "correct" dvd region for the region they were supposed to be sold to. Capitalism can be scary.</div>
<div>
So, you can select a region, or just leave it as it is.</div>
<div>
More info at /usr/share/doc/regionset/README</div>
<div>
<br /></div>
<div>
Finally, let's try playing something</div>
<div>
$ mpv dvd://1</div>
<div>
<br /></div>
<div>
And you can check your dvd contents with</div>
<div>
$ lsdvd</div>
<div>
<br /></div>
<div>
I'm sure you can also click around with your favorite desktop environment and find a way to play your dvd as well.</div>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-35236171370180489992020-03-31T11:55:00.005+09:002020-03-31T11:55:59.506+09:00flask basic auth super simpleI wrote a super simple basic auth using flask app that does all the authentication in the app side.<br />
<br />
<a href="https://gitlab.com/rikijpn/flask_simple_basic_auth">https://gitlab.com/rikijpn/flask_simple_basic_auth</a><br />
<br />
Don't ask me why... But hey, it works.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-49955375085672067662020-03-31T04:54:00.000+09:002020-03-31T04:56:54.786+09:00Using microsoft graph API to book things in outlook<h2>
Quick notice</h2>
I hate microsoft. Soooo much. And its API even more. I hated everything about using it, and having to read its very hard to understand documentation.<br />
But it's the lesser evil to actually having to use ms outlook to book meeting rooms in an environment non ms free. The real solution is to get rid of microsoft and everything microsoft in your office.<br />
<h2>
Purpose</h2>
<div>
My office has these meeting rooms, which are always occupied. And it's very hard to make a reservation. I hate ms outlook too much to have to do this every day manually, so I checked out the ms "graph API" (in case you wonder, it's not to make "graphs", it's just the stupid name they use for their office API).<br />
Now I can get all the meeting rooms I need daily for all my team without having to move a finger.</div>
<h2>
Code</h2>
<div>
The actual script I use is here: <a href="https://gitlab.com/rikijpn/ugly_ms_outlook_appointment_taker">https://gitlab.com/rikijpn/ugly_ms_outlook_appointment_taker</a><br />
I'm obviously removing all the tokens though.</div>
<div>
It's in python, running daily. So you can just put this in cron, so as soon as they let you book meeting rooms (like 30, 60 days after the current day's specific time) you'be able to send the appointment request.</div>
<h2>
Preparation (create an "app")</h2>
<br />
1. create app (in your azure portal)
<br />
In the left-hand navigation pane, select the Azure Active Directory service, and then select App registrations → New registration.<br />
<a href="https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps">https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps</a>
<br />
2. in azure, select Manage → API Permissions → app (Microsoft APIs → Microsoft Graph → Delegated permissions)<br />
Make sure you have the following permissions:<br />
<pre>Calendars.ReadWrite
User.Read
offline_access
</pre>
<br />
3. in azure, select Manage → Authentication → "Default client type" → "Treat application as public client"<br />
4. in azure, select Manage → Authentication → "Supported account types" → "Accounts in this organizational directory only (this will probably have your company/school name here)"<br />
5. get consent
<br />
<a href="https://login.microsoftonline.com/$%7Btenant_id%7D/oauth2/v2.0/authorize?client_id=${client_id}&response_type=code&response_mode=query&state=12345&scope=offline_access%20user.read%20Calendars.ReadWrite">https://login.microsoftonline.com/${tenant_id}/oauth2/v2.0/authorize?client_id=${client_id}&response_type=code&response_mode=query&state=12345&scope=offline_access%20user.read%20Calendars.ReadWrite</a>
<br />
The "tenant_id" here is just whatever you already have in your url when looking azure, a long senseless string, that is common through all the pages for your organization.<br />
"client_id" will be shown in your "app"'s top/info page.<br />
You can just ignore the rest and use it as it is.<br />
<br />
Put this in your browser, and you'll be sent to a blank page by default. Check the URL in your browser, and you'll see there will be an "access" and "refresh" token.<br />
<h2>
Usage</h2>
And that's pretty much it. Now you are ready to do some tests.<br />
I'm posting the test code (emacs's restclient-mode stuff) in the same gitlab repo: <a href="https://gitlab.com/rikijpn/ugly_ms_outlook_appointment_taker/-/blob/master/sample_requests.txt">https://gitlab.com/rikijpn/ugly_ms_outlook_appointment_taker/-/blob/master/sample_requests.txt</a><br />
In short, every time you want to use it, you'll have to use your "refresh_token" to create a new "access_token", which lasts very shortly (a couple of mins? one hour? don't remember).<br />
<br />
<br />
One thing though, is that every time you change your password, you'll have to do get consent again:<br />
<a href="https://login.microsoftonline.com/$%7Btenant_id%7D/oauth2/v2.0/authorize?client_id=${client_id}&response_type=code&response_mode=query&state=12345&scope=offline_access%20user.read%20Calendars.ReadWrite">https://login.microsoftonline.com/${tenant_id}/oauth2/v2.0/authorize?client_id=${client_id}&response_type=code&response_mode=query&state=12345&scope=offline_access%20user.read%20Calendars.ReadWrite</a><br />
<div>
<br /></div>
And update your tokens.<br />
<br />
When you make a meeting room reservation (appointment, using the meeting room as a "resource"), you'll get a 200 status code if your auth has no issues. But that DOES NOT mean your meeting room was acknowledged, and that you got the meeting room. It just means the damn ms outlook server got your request. The pain.<br />
You'll get an e-mail with the approval/denial response a bit after your request is processed. I think you can also check with the API somehow.<br />
<h2>
Final thoughts</h2>
Really ugly documentation. That's how I've described pretty much all ms documentation in the past, and how I still do for everything about this "graph API" thing.<br />
It's like, they don't really want you to be able to use it.<br />
The token usage is frankly not bad, seems like a good (secure and clear) authorization and all, most likely thanks to its usage of OAuth2.0 and JWT.<br />
<br />rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-26298038145881333542020-03-26T23:30:00.001+09:002020-03-26T23:30:29.947+09:00ssh tunnelsI love ssh tunnels.<br />
<br />
Working in an office, with a super strict data center, they make my work so much easier.<br />
<br />
Basically an ssh tunnel is a way to put a local port in a remote server, and vice-versa.<br />
<br />
Scenarios it might be useful:<br />
<br />
<ol>
<li>You have a bastion/jump server that you need to connect to do EVERYTHING. It makes sense to just have one main ssh session established, and just connect with the same dynamic proxy tunnel to all the rest of the servers stepping through, without having to connect again each time.<br /><br />Situation:<br />local machine TO jump server<br />local machine TO jump server, jump server TO server only accessible by jump server1<br />local machine TO jump server, jump server TO server only accessible by jump server2<br />...<br />(above, you need to ssh to the jump server, and from then, ssh to the remote server, every time.)<br /><br />Solution:<br />local machine TO jump server<br />local machine ( through tunnel ) server only accessible by jump server1<br />local machine ( through tunnel ) server only accessible by jump server2<br />...<br />(above, you just open one session to the jump server. And then can ssh directly from your local box to the remote servers through the tunnel)</li>
<li>You can ssh from your local machine TO a server, but not the other way around (due to firewall, being behind a router, etc)<br /><br />Situation:<br />local machine TO remote server = OK<br />remote server TO local machine = NO<br /><br />Solution:<br />local machine TO remote server (remote forward:9001 to localhost:22)<br />remote server TO remote server port 22 (=local machine's ssh port = local machine)</li>
<li>You want to share a port/server only accessible from the office, to a remote server. Let's say, your intra network's GIT server, to a server. Or your mail server, local printer, etc.<br /><br />Situation:<br />local machine TO intra git = OK<br />remote server TO intra anything = NO<br /><br />Solution:<br />local machine TO remote server (remote forward RANDOM_PORT to intra git server's SSH PORT)<br />remote server TO remote server's RANDOM_PORT (goes through your local box, and to intra's git server)</li>
<li>Debugging your flask DEV env on port 5000, and your firewall doesn't let you access it directly.<br /><br />Situation:<br />local machine TO remote server's port 5000 = NO<br />local machine TO remote server = OK (ssh)<br />remote server TO remote server's port 5000 = OK<br /><br />Solution:<br />local machine TO remote server (local forward 5000 to localhost:5000)<br />local machine TO local machine's port 5000 ( = remote server's port 5000)</li>
</ol>
<div>
<br /></div>
<div>
These are just some simple examples.</div>
<div>
You can also kind of nest them! One ssh tunnel to another, and another... and sometimes it gives you a bit of a headache.</div>
<div>
My .ssh/config file is huge due to this, but it beats the alternative of not being able to debug a flask development instance live with your browser, for example.</div>
<div>
<br /></div>
<div>
<br /></div>
<h2>
What is dynamic/local/remote forwarding?</h2>
<div>
In short:</div>
<div>
Dynamic tunnels act as SOCKS servers. This means you can ssh connect with the nc command sending all the stream directly. When you set up dynamic forwarding for a host, your local machine dynamic forwarding port will act as a SOCKS server.</div>
<div>
<br /></div>
<div>
local forwarding = the port will be on your local machine, pointing to somewhere in the remote server</div>
<div>
<br /></div>
<div>
remote forwarding = the port will be in the remote server, pointing to somewhere on your local machine</div>
<div>
<br /></div>
<div>
Easy, right?</div>
<h2>
So, how to do you this?</h2>
<div>
<br /></div>
<div>
I think you can do this in windows terminals too, and even macs. But This is GNU/Linux blog, so I'll be talking only about the ssh command and its config.</div>
<div>
<br /></div>
<div>
You can do all this with arguments to the ssh command, but I was never able to remember those. So I just edit my ~/.ssh/config file for all these settings, and that's the way I'll be introducing.</div>
<div>
<br /></div>
<div>
All settings below will be in your <b>local machine</b>'s ~/.ssh/config.</div>
<h3>
Let's create a dynamic tunnel</h3>
<div>
<br /></div>
<pre>Host some-jump-server.net
# these two aren't really necessary, I just like them
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
DynamicForward 5555
</pre>
<br />
So, you just "ssh some-jump-server.net" in one terminal, and keep that session open.<br />
To access a server only accessible from the jump server, you just add that server to your ~/.ssh/config, defining it as a server that needs to go through that proxy, or you can also like create a script for the same purpose.<br />
<br />
This is how I'd register the server:<br />
<br />
<pre>Host some-server.my-subnet
ProxyCommand nc -x 127.0.0.1:5555 %h %p
</pre>
<br />
Then you just "ssh some-server.my-subnet". And in one ssh, you should be accessing the remote server.<br />
You'll probably be needing an ssh agent if you plan to do this though, but that's a topic for another post.<br />
<br />
Also, notice you don't need to add hosts one by one, you can set complete sub-domains, or just "all" with the "*" wildcard (next to the "Host" keyword). "*.my-subnet" for example, would set this dynamic proxy port for all servers you try to ssh that end with ".my-subnet".<br />
<br />
<h3>
Let's try a localforward</h3>
Local Fowards open a port in your local machine, to somewhere pointing on the remote server.<br />
Let's use the flask development port (5000) for example, that you can only see in your server, and can't access directly from anywhere else.<br />
<br />
<pre>Host some-server-with-my-flask-dev.my-subnet
LocalForward 8559 localhost:5000
</pre>
<br />
So, now you can just put your in local machine's browser the url http://localhost:8559/, and you'll be accessing your remote host's port 5000, yay. No need to proxy or anything, just ssh.<br />
<h3>
And, let's try a remoteforward</h3>
The last one, remote forward, opens a port in the remote server, pointing somewhere in your local machine.<br />
<br />
<pre>Host some-server-that-needs-intra-stuff.my-subnet
# pointing your local git server for example
RemoteForward 5757 192.168.11.1:7999
# or to your tiny proxy http proxy, in case you need to access intra web
RemoteForward 8999 localhost:8888
</pre>
<pre>
</pre>
So, if you ssh to some-server-that-needs-intra-stuff.my-subnet, you'll be able to see with "netstat -atn|grep LISTEN" for example, that the ports 5757 and 8999 are open. And they'll be pointing to your local machine's 7999 and 8888 ports respectively. In this example, they're your local git server, and http proxy. So virtually, you could be accessing the same INTRA web from a DC that shouldn't have access at all! yay! Welcome to the wonderful world of security risks as well, be careful!<br />
<br />
<br />
In conclusion, ssh tunneling is fun. Remember you can also have multiple remote and local forwards in each host block, and you mix them as well. It can be quite a mess! But sooo damn useful.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-86428481909036555362020-03-25T16:03:00.001+09:002020-05-14T20:20:06.621+09:00using synergy to share your keyboard and mouse with another computer<h2>
What's Synergy?</h2>
An awesome program that allows you to share your mouse/keyboard over the network with another computer.<br />
You basically need to have the "server" installed on the computer your keyboard/mouse are connected to, and a "client" to the other one.<br />
<br />
<br />
I use this at the office, where I'm sadly forced to use ms windows, and connect it to my GNU/Linux box.<br />
I also use it at home when I bring my office laptop sometimes, and just do all the input with my home keyboard/mouse, as I'm a happyhacking keyboard fan.<br />
<br />
<h2>
How to install</h2>
<div>
On Windows: google for "windows synergyc 1.8.8", for the windows one, and install that</div>
<div>
On GNU/Linux Debian-family: sudo apt-get install synergy</div>
<div><br /></div><h3 style="text-align: left;">in Debian 10</h3><div>As of debian 10, synergy is an outdated package apparently. I guess it wasn't very popular...</div><div><br /></div><div>But you can install it from the stretch's repo directly, like this:</div><div><div><br /></div><div><br /></div><div>sudo apt-get install libcrypto++6 libqt4-network libqtcore4 libqtgui4</div><div>wget http://ftp.jp.debian.org/debian/pool/main/s/synergy/synergy_1.4.16-2_amd64.deb #just find this deb file in whatever mirror you prefer</div></div><div><div>sudo dpkg -i synergy_1.4.16-2_amd64.deb </div></div><div><br /><br /></div>
<h2>
How to setup</h2>
<h4>
in ms windows (client)</h4>
<div>
Just make a "shortcut" with something like "C:\Synergy\synergyc.exe XX.XX.XX.XX" (your server's IP)</div>
<div>
<br /></div>
<h4>
in GNU/Linux</h4>
<div>
create a file /etc/synergy.conf with these contents:</div>
<div>
<br /></div>
<div>
<div>
<br /></div>
<div>
section: screens</div>
<div>
random_name_for_your_server_box:</div>
<div>
random_name_for_your_client_box:</div>
<div>
end</div>
<div>
</div>
<div>
section: links</div>
<div>
random_name_for_your_client_box:</div>
<div>
left = random_name_for_your_server_box</div>
<div>
random_name_for_your_server_box:</div>
<div>
right = random_name_for_your_client_box</div>
<div>
end</div>
<div>
<br /></div>
</div>
<div>
<br /></div>
<div>
You'll also need to put your "random_name_for_your_server_box" name in /etc/hosts.</div>
<div>
<br /></div>
<h2>
How to use</h2>
<div>
1. in windows: double click the "shortcut" thing</div>
<div>
2. in GNU/Linux, open a terminal, and type "synergys"</div>
<div>
<br /></div>
<div>
The client terminal should show some debug messages, like "server connected!" and stuff like that.</div>
<div>
Just move your mouse pointer and see it go from your server screen to your client's.</div>
<div>
<br /></div>
<div>
I'm surprised really so few people know about this and end up buying more keyboards, or worst, using a bluetooth one just to be able to switch the input (facepalm). sshing is also super useful and all, but sometimes you use a crappy OS like ms windows in which that's just not possible, or just want to use it on its same screen.</div>
<div>
<br /></div>
<div>
You can also use this in a better environment, GNU/Linux with GNU/Linux.</div>
<div>
Don't get me started about macs.</div>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-88339278454939411192019-12-05T18:56:00.001+09:002019-12-05T18:56:46.876+09:00connecting to wifi in a self created certificate enterprise environmentBasically, my office environment needs me to connect with a self signed certificate for our wifi, and in GNU/Linux, this was a lot harder than I expected.<br />
<br />
In short, you can't use the default/easy to use nmcli. But you have to set your username/password directly in the wpa_supplicant conf, and just connect directly with the wpa_supplicant program.<br />
<br />
<br />
<h2>
This is what works</h2>
you can get the list of SSIDs available by<br />
<br />
<pre>nmcli device wifi list
</pre>
<br />
<pre># 1 make this file
################################################################################
# /etc/wpa_supplicant.conf
network={
ssid="XXX"
key_mgmt=WPA-EAP
identity="your_username"
password="your_password"
}
################################################################################
# enable wpa_supplicant (connect to wifi)
wpa_supplicant -Dnl80211 -iwlx00018e8e09f2 -c /etc/wpa_supplicant.conf
#then you have to do dhclient to get an IP
sudo dhclient wlx00018e8e09f2
</pre>
<div>
<br /></div>
<br />
<h2>
This is what I also needed to do for my crappy wifi usb card</h2>
<br />
ended up creating a script to enable my crappy usb wifi<br />
<pre>################################################################################
modprobe rt2800usb
echo "0789 0168" > /sys/bus/usb/drivers/rt2800usb/new_id
# wpa_supplicant -B -Dnl80211 -iwlx00018e8e09f2 -c /etc/wpa_supplicant.conf &
wpa_supplicant -Dnl80211 -iwlx00018e8e09f2 -c /etc/wpa_supplicant.conf &
sleep 3
echo killing initial wpa_supplicant
killall wpa_supplicant
sleep 3
echo starting final wpa_supplicant
</pre>
<br />
<h2>
Some of the things I tried and didn't work</h2>
<br />
nmcli doesn't work as it NEEDS to do cacert validation, apparently that
<br />
can't be disabled on the program...
<br />
<br />
this also didn't work
<br />
<pre>dconf write /org/gnome-nm-applet/eap/70b2f88f-1c43-45e0-acd5-276f66bd0b98/ignore-phase2-ca-cert false
dconf write /org/gnome-nm-applet/eap/70b2f88f-1c43-45e0-acd5-276f66bd0b98/ignore-ca-cert true
</pre>
<br />
this works for non enterprise wifi devices I guess<br />
<pre>nmcli device wifi connect XXX username XXX password XXX
</pre>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-10912396636780128232019-12-05T18:38:00.001+09:002019-12-05T18:38:54.101+09:00cross cable connection<h2>
How to connect two GNU/Linux computers with a cross cable</h2>
<br />
on box1<br />
<pre>sudo ifconfig enp1s0 192.168.0.1 netmask 255.255.255.0 up
</pre>
<br />
<br />
on box2<br />
<pre>sudo ifconfig enp1s0 192.168.0.2 netmask 255.255.255.0 up
</pre>
<div>
here I'm using enp1s0 as my eth0 interface name, it might just be eth0 on yours. You can check with whatever you get as LAN by the "ip a show" command<br />
<br />
This is the cable I got, but any cross cable should do</div>
<div>
<br /></div>
<table border="0" cellpadding="0" cellspacing="0"><tbody>
<tr><td><div style="background-color: white; border-radius: 0.75rem; border: 1px solid #95a5a6; margin: 0px; overflow: hidden; padding: 5px; text-align: center; width: 504px;">
<table><tbody>
<tr><td style="width: 240px;"><a href="https://hb.afl.rakuten.co.jp/hgc/19c8de54.27c3960e.19c8de55.9bbc2590/?pc=https%3A%2F%2Fitem.rakuten.co.jp%2Fesupply%2Fkb-t5y-02lbcn%2F&m=http%3A%2F%2Fm.rakuten.co.jp%2Fesupply%2Fi%2F10064884%2F&link_type=picttext&ut=eyJwYWdlIjoiaXRlbSIsInR5cGUiOiJwaWN0dGV4dCIsInNpemUiOiIyNDB4MjQwIiwibmFtIjoxLCJuYW1wIjoicmlnaHQiLCJjb20iOjEsImNvbXAiOiJkb3duIiwicHJpY2UiOjEsImJvciI6MSwiY29sIjoxLCJiYnRuIjoxLCJwcm9kIjowfQ%3D%3D" rel="nofollow noopener noreferrer" style="word-wrap: break-word;" target="_blank"><img alt="[商品価格に関しましては、リンクが作成された時点と現時点で情報が変更されている場合がございます。]" border="0" src="https://hbb.afl.rakuten.co.jp/hgb/19c8de54.27c3960e.19c8de55.9bbc2590/?me_id=1208050&item_id=10064884&m=https%3A%2F%2Fthumbnail.image.rakuten.co.jp%2F%400_mall%2Fesupply%2Fcabinet%2Fproduct_k_4%2Fkb-t5y-02lbcn_1.jpg%3F_ex%3D80x80&pc=https%3A%2F%2Fthumbnail.image.rakuten.co.jp%2F%400_mall%2Fesupply%2Fcabinet%2Fproduct_k_4%2Fkb-t5y-02lbcn_1.jpg%3F_ex%3D240x240&s=240x240&t=picttext" style="margin: 2px;" title="[商品価格に関しましては、リンクが作成された時点と現時点で情報が変更されている場合がございます。]" /></a></td><td style="vertical-align: top; width: 248px;"><div style="font-size: 12px; line-height: 1.4em; margin: 0px; padding: 2px 6px; text-align: left; word-wrap: break-word;">
<a href="https://hb.afl.rakuten.co.jp/hgc/19c8de54.27c3960e.19c8de55.9bbc2590/?pc=https%3A%2F%2Fitem.rakuten.co.jp%2Fesupply%2Fkb-t5y-02lbcn%2F&m=http%3A%2F%2Fm.rakuten.co.jp%2Fesupply%2Fi%2F10064884%2F&link_type=picttext&ut=eyJwYWdlIjoiaXRlbSIsInR5cGUiOiJwaWN0dGV4dCIsInNpemUiOiIyNDB4MjQwIiwibmFtIjoxLCJuYW1wIjoicmlnaHQiLCJjb20iOjEsImNvbXAiOiJkb3duIiwicHJpY2UiOjEsImJvciI6MSwiY29sIjoxLCJiYnRuIjoxLCJwcm9kIjowfQ%3D%3D" rel="nofollow noopener noreferrer" style="word-wrap: break-word;" target="_blank">Cat 5eより線クロスケーブル(2m・ライトブルー) KB-T5Y-02LBCN サンワサプライ【ネコポス対応】</a><br />価格:480円(税込、送料別) <span style="color: #bbbbbb;">(2019/12/5時点)</span></div>
<div style="margin: 10px;">
<a href="https://hb.afl.rakuten.co.jp/hgc/19c8de54.27c3960e.19c8de55.9bbc2590/?pc=https%3A%2F%2Fitem.rakuten.co.jp%2Fesupply%2Fkb-t5y-02lbcn%2F&m=http%3A%2F%2Fm.rakuten.co.jp%2Fesupply%2Fi%2F10064884%2F&link_type=picttext&ut=eyJwYWdlIjoiaXRlbSIsInR5cGUiOiJwaWN0dGV4dCIsInNpemUiOiIyNDB4MjQwIiwibmFtIjoxLCJuYW1wIjoicmlnaHQiLCJjb20iOjEsImNvbXAiOiJkb3duIiwicHJpY2UiOjEsImJvciI6MSwiY29sIjoxLCJiYnRuIjoxLCJwcm9kIjowfQ%3D%3D" rel="nofollow noopener noreferrer" style="word-wrap: break-word;" target="_blank"><img src="https://static.affiliate.rakuten.co.jp/makelink/rl.svg" style="float: left; margin-top: 0; max-height: 27px; width: auto;" /></a><a href="https://hb.afl.rakuten.co.jp/hgc/19c8de54.27c3960e.19c8de55.9bbc2590/?pc=https%3A%2F%2Fitem.rakuten.co.jp%2Fesupply%2Fkb-t5y-02lbcn%2F%3Fscid%3Daf_pc_bbtn&m=http%3A%2F%2Fm.rakuten.co.jp%2Fesupply%2Fi%2F10064884%2F%3Fscid%3Daf_pc_bbtn&link_type=picttext&ut=eyJwYWdlIjoiaXRlbSIsInR5cGUiOiJwaWN0dGV4dCIsInNpemUiOiIyNDB4MjQwIiwibmFtIjoxLCJuYW1wIjoicmlnaHQiLCJjb20iOjEsImNvbXAiOiJkb3duIiwicHJpY2UiOjEsImJvciI6MSwiY29sIjoxLCJiYnRuIjoxLCJwcm9kIjowfQ==" rel="nofollow noopener noreferrer" style="word-wrap: break-word;" target="_blank"></a><br />
<div style="background-color: #bf0000; border-radius: 16px; color: white; cursor: pointer; float: right; font-size: 12px; font-weight: 500; height: 27px; line-height: 27px; margin-left: 1px; padding: 0 12px; text-align: center; width: 41%;">
<a href="https://hb.afl.rakuten.co.jp/hgc/19c8de54.27c3960e.19c8de55.9bbc2590/?pc=https%3A%2F%2Fitem.rakuten.co.jp%2Fesupply%2Fkb-t5y-02lbcn%2F%3Fscid%3Daf_pc_bbtn&m=http%3A%2F%2Fm.rakuten.co.jp%2Fesupply%2Fi%2F10064884%2F%3Fscid%3Daf_pc_bbtn&link_type=picttext&ut=eyJwYWdlIjoiaXRlbSIsInR5cGUiOiJwaWN0dGV4dCIsInNpemUiOiIyNDB4MjQwIiwibmFtIjoxLCJuYW1wIjoicmlnaHQiLCJjb20iOjEsImNvbXAiOiJkb3duIiwicHJpY2UiOjEsImJvciI6MSwiY29sIjoxLCJiYnRuIjoxLCJwcm9kIjowfQ==" rel="nofollow noopener noreferrer" style="word-wrap: break-word;" target="_blank">楽天で購入</a></div>
</div>
</td></tr>
<tr></tr>
</tbody></table>
</div>
<br />
<div style="color: black; font-size: 12px; line-height: 1.4em; margin: 5px; word-wrap: break-word;">
</div>
</td></tr>
</tbody></table>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-48737063630911752772018-04-04T23:36:00.002+09:002023-10-24T11:25:25.367+09:00Macross news in RSS<h2>
What I did</h2>
<div>
An RSS newsfeed you can use with your favorite RSS reader to checkout Macross related news (it's all in Japanese though... based on the site <a href="http://macross.jp/news/list.php">Macross Portal (News)</a>)</div>
<div>
<br /></div>
<h2>
RSS feeds</h2>
<div>
<br /></div>
<div>
<a href="http://sun4.gmobb.jp/rikijpn/macross_news_rss.xml">macross_news_rss.xml (full, sub pages are made into RSS CDATA contents)</a></div>
<div><br /></div>
<h2>
Source</h2>
https://gitlab.com/rikijpn/macross-news-rss<br />
( I deleted my github account, f*ck microsoft!)<br />
<br />
<h2>
A bit about RSS</h2>
<div>
I love RSS.</div>
<div>
I read lots of RSS feeds with emacs gnus, together with my mail at work, and it's just so much better than having to go to each website to look for news and stuffs. Due to my Free Software beliefs I don't use SNS/microblogging services very often.</div>
<div>
You could even use it for twitter and stuffs, but I'd rather hope someday there will be a world where our main SNS services will be decentralized as diaspora... sorry, that's another topic.</div>
<div>
<br /></div>
<h3>
A bit about this code</h3>
<div>
I don't know why I even bothered using that lxml library... it'd have saved me so much time to just write my own xml-like class. But in the long term I guess lxml is a lot better to maintain real (large) xml.</div>
<div>
RSS isn't really made of a super complex XML, but a rather small, with a lot of commonly used fields defined one.</div>
<div>
<br /></div>
<div>
The simple version is my starting one, which is good enough for small things. But this site usually has some pics I end up wanting to see, so clicking on the "link" part very often. I'd stand that for RSS made by the official site, but since it's my whole thing, I decided to add a "full" version, with the complete page as the "contents" of each news, so I very rarely need to actually access the original site anymore, yay.</div>
<div>
<br /></div>
<div>
I put the scripts on cron to run daily, and to do the sync with the web server.</div>
<div>
<br /></div>
<div>
The hardest part was knowing how to write RSS... since apparently there are lots of things that aren't defined-per-se, but are common used like that, etc.</div>
<div>
I ended up mimicking some site's RSS feed. Which gave me the most important hints I needed:</div>
<div>
1. You need to use the "<content:encoded>" tags instead of "<description>" for CDATA (html code) including source.</description></content:encoded></div>
<div>
2. CDATA let's you put HTML! Copying sections from web pages becomes a lot easier that way...</div>
<div>
3. You need to declare the "encoded" Namespace (some kind of XML terminology I still don't really understand, but thanks to the lxml library I had to read a lot of in order to just make those little content:encoded tags!)</div>
<div>
<br /></div>
<div>
That's pretty much it.</div>
<div>
All the rest you can just read in <a href="https://www.w3schools.com/xml/xml_rss.asp">w3schools</a>. They even have an XML namespaces section!</div>
<div>
<br /></div>
<div>
For the simple version you don't even need more than what w3schools has in their tutorials.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The code is a bit messy (especially the "full version" one). But basically I have one class to fetch the data, and another to make an RSS(XML) object.</div>
<div>
So I fetch the main news titles page, and make an RSS feed out of it, voila.</div>
<div>
<br /></div>
<div>
In the "full version" one, as I'm fetching not only the titles, but the contents of the pages behind those titles, I'm saving the data every time, so only new titles' sub pages are fetched and not all pages every time.</div>
<div>
Plus I added a description fetching class pretty similar to the one for the titles, but that gets the main HTML part I want to have as contents, and after checking the new news have no contents yet, add them there.</div>
<div>
<br /></div>
<div>
<br /></div>
<h2>
That's it</h2>
<div>
Well, I hope I never forget about another concert or new Valkyrie sales after this. You're welcome to use the RSS if you're a macross fan too.</div>
<div>
I'm kinda regretting not having used lisp to write this thing since that would have probably been faster... but I guess I should be able to write some python too and all.</div>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-41622713532097507202017-10-19T22:14:00.002+09:002017-10-19T22:14:49.340+09:00LINE timeline video download If you live in Japan you'll probably be using LINE to chat with everyone.<br />
<br />
Here's how you can download movies from your timeline (your android won't let you without rooting it...).<br />
<br />
You'll need:<br />
1. chrome or chromium<br />
2. the chrome line app? (maybe not needed)<br />
3. some html/javascript knowledge?<br />
<br />
This is what you do:<br />
1. Login to your LINE account using the chrome LINE app<br />
2. Click on the timeline icon on the app, this will take you to somewhere like https://timeline.line.me<br />
3. Login to the timeline screeen<br />
4. Do Ctrl-Shift-J to get the javascript console<br />
<br />
5-a. You can inspect the DOM html code until the vid you want is highlighted, and click until you find the "video" html tag, then just copy paste the src attribute's value (an mp4 vid url)<br />
<br />
5-b You can also just get all the vids in the page by typing the javascript:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> document.getElementsByTagName('video')</span><br />
<br />
if it's only one, then<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> document.getElementsByTagName('video')[0].src</span><br />
<br />
should give you the url for the vid, you just copy paste this<br />
<br />
6. paste in browser's omnibar and start downloading!<br />
<br />
<br />rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-3601193589821719332017-08-19T21:47:00.003+09:002017-08-19T21:47:34.641+09:00downloading torrents in GNU/linux (magnet URIs)Torrents have changed so much...<br />
<br />
Nowadays they're greatly distributed in magnet:XXX-looking URIs.<br />
Check more about that in http://www.vuze.com/about-torrents/magnet-links.<br />
I guess it's better than getting lots of torrent files.<br />
<br />
Anyway, to be able to open and use these files, here's my favorite torrent client and how to use it.<br />
It's utorrent!<br />
<br />
http://www.utorrent.com/downloads/linux<br />
<br />
1. Get the utorrent server (it's a server because you can actually access it from the internet too, or your LAN, but you can just use it on localhost as a normal client).<br />
2. cd to wherever the file "utserver" is, and run "./utserver -settingspath &"<br />
3. login to http://localhost:8080/ (user = admin, blank password by default)<br />
4. click the chain looking icon to add a random torrent URL and you're done.<br />
<br />
For non GUI related stuffs I liked aria2c, but it stopped working... lots of data are now being encrypted and everything, kinda going out of the original scope of a bittorrent client, which has some good and bad sides.<br />
<br />
Anyway, have fun with utorrent.<br />
<br />rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-64285920019555017222017-08-07T21:01:00.001+09:002017-08-07T21:01:58.217+09:00common lisp getting drakma working from default install<h2>
Basic clisp setup to get drakma working</h2>
<div>
<br /></div>
<div>
drakma is a lisp html client.</div>
<div>
<br /></div>
<div>
I'm using GNU CLISP 2.49.</div>
<div>
<br /></div>
<div>
If you just try to install it with the default asdf and quicklisp installed under it on debian 8 it won't work...</div>
<div>
<br /></div>
<div>
Anyway,</div>
<div>
1. Install asdf's latest version https://common-lisp.net/project/asdf/ (I have 3.1.4) </div>
<div>
This means just downloading the source, and doing make mostly.</div>
<div>
2. load your new asdf every time by adding it to your ~/clisprc.lisp</div>
<div>
something like</div>
<div>
<code>(load "/yourhomedir/bin/lisp/clisp/asdf/build/asdf.lisp")</code></div>
<div>
3. Install quicklisp https://www.quicklisp.org/beta/#installation</div>
<div>
This is mostly downloading the install file, and doing </div>
<div>
<div>
<code>(load "/yourhomedir/bin/lisp/clisp/install_ql/quicklisp.lisp")</code></div>
<div>
<code>(ql:add-to-init-file)</code></div>
</div>
<div>
<br /></div>
<div>
4. Then you're ready to install drakma</div>
<div>
<div>
<code>(ql:quickload :drakma)
(drakma:http-request "https://www.debian.org/" :connection-timeout 'nil)
</code>
</div>
</div>
<div>
<div>
</div>
<div>
(in common lisp you need the :connection-timeout 'nil thing or it won't work... that's not very nice)</div>
<div>
<br /></div>
<div>
And you should be getting something the requested page on stdout:</div>
<code>
</code>
<br />
<div>
...</div>
</div>
<div>
...!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd<br />
...<br />
<br />
<br /></div>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-28410643459092258042017-08-07T20:27:00.000+09:002017-08-07T20:27:32.418+09:00debian change display output interface from vga to dvi<h2>
Updating your X output interface</h2>
<div>
<br /></div>
<div>
I have an nvidia video card, and I'm finally switching from VGA to DVI... I didn't even know I had that port till a few days ago.</div>
<div>
I'm not adding a new display for dual/triple display, but just switching the cables for the same one and only display I have.</div>
<div>
<br /></div>
<div>
1. Plug in the DVI cable to your monitor and video card.</div>
<div>
2. When you do the xrandr command, you should be seeing something like:</div>
<code>
<span style="font-family: monospace;">$ xrandr</span><br />
<span style="font-family: monospace;">Screen 0: minimum 8 x 8, current 1440 x 900, maximum 16384 x 16384</span><br />
<span style="font-family: monospace;">DVI-I-0 disconnected primary (normal left inverted right x axis y axis)</span><br />
<span style="font-family: monospace;">VGA-0 connected 1440x900+0+0 (normal left inverted right x axis y axis) 410mm x 260mm</span><br />
<span style="font-family: monospace;"> 1440x900 59.89*+ 74.98 </span><br />
<span style="font-family: monospace;"> 1400x1050 74.87 </span><br />
<span style="font-family: monospace;"> 1280x1024 75.02 60.02 </span><br />
<span style="font-family: monospace;"> 1152x864 75.00 </span><br />
<span style="font-family: monospace;"> 1024x768 75.03 70.07 66.00 60.00 </span><br />
<span style="font-family: monospace;"> 800x600 75.00 72.19 60.32 56.25 </span><br />
<span style="font-family: monospace;"> 640x480 75.00 72.81 65.99 59.94 </span><br />
<span style="font-family: monospace;">DVI-I-1 connected (normal left inverted right x axis y axis)</span><br />
<span style="font-family: monospace;"> 1440x900 59.89 + 74.98 </span><br />
<span style="font-family: monospace;"> 1400x1050 74.87 </span><br />
<span style="font-family: monospace;"> 1280x1024 75.02 60.02 </span><br />
<span style="font-family: monospace;"> 1152x864 75.00 </span><br />
<span style="font-family: monospace;"> 1024x768 75.03 70.07 66.00 60.00 </span><br />
<span style="font-family: monospace;"> 800x600 75.00 72.19 60.32 56.25 </span><br />
<span style="font-family: monospace;"> 640x480 75.00 72.81 65.99 59.94 </span><br />
<span style="font-family: monospace;">HDMI-0 disconnected (normal left inverted right x axis y axis)</span><br />
</code><br />
<span style="font-family: monospace;"><br /></span>
<span style="font-family: monospace;">3. Here you see the VGA-0 (your current display), and DVI-I-1 (your new one) are connected.</span><br />
<span style="font-family: monospace;">The only thing to do next is:</span><br />
<span style="font-family: monospace;"><br /></span>
<code><span style="font-family: monospace;">xrandr --output DVI-I-1 --auto</span></code><br />
<span style="font-family: monospace;"><br /></span>
<span style="font-family: monospace;"><br /></span><span style="font-family: monospace;">4. Your monitor should blink for a moment. Then you're probably ready to go! Disconnect your VGA cable, and if you still have something on your screen, it's working, yay.</span><br />
<span style="font-family: monospace;"><br /></span>
<span style="font-family: monospace;">5. Disconnect your old VGA cable, and give it a good burial.</span><br />
<span style="font-family: monospace;"><br /></span>rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-59251909558861038532017-06-04T12:36:00.002+09:002017-06-04T12:36:38.137+09:00fixing B-frames ('packed B-frames') vids<h2>
mplayer warning</h2>
[mpeg4 @ 0xb7411fe0]Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using a tool like VirtualDub or avidemux to fix it.<br />
<div>
<br /></div>
<div>
You'd get this warning, and video wouldn't be playable (it'd be super slow in the video stream, while the audio is OK).</div>
<h2>
Suggested fix</h2>
<div>
ffmpeg -i input_vid.avi -codec copy -bsf:v mpeg4_unpack_bframes output_vid.avi</div>
<div>
<br /></div>
<div>
(didn't work, still was super slow)</div>
<div>
<br /></div>
<h2>
Work-around</h2>
<div>
You'll need: avidemux, ffmpeg</div>
<div>
<br /></div>
<div>
Steps</div>
<div>
1. convert file to avi (mine were in ogm format)</div>
<div>
<br /></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">ffmpeg -i $INPUT_FILE -codec copy $OUTPUT_FILE</span></div>
<div>
<br /></div>
<div>
2. separate audio</div>
<div>
<br /></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">ffmpeg -i $INPUT_FILE -vn $OUTPUT_FILE</span></div>
<div>
<br /></div>
<div>
3. open the avi file on avidemux -> choose audio -> disable sound tracks</div>
<div>
<br /></div>
<div>
4. merge audio and video in one final vid</div>
<div>
<br /></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">ffmpeg -i $VID_FILE -i $AUDIO_FILE -codec copy -shortest $OUTPUT_FILE</span></div>
<div>
<br /></div>
<div>
You can then just remove all remaining files created during the process and just the one on step 4. It'll still show the warning, but the video will be watchable (it was for me at least).</div>
<div>
<br /></div>
rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com0tag:blogger.com,1999:blog-8235998570069782586.post-54277264593354258982015-11-15T21:19:00.001+09:002015-11-15T21:19:23.262+09:00SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) fix on debian 8I'm using:
<br />
<code>
<br />
Debian GNU/Linux 8
<br />
Python 2.7.9
<br />
openssl 1.0.1k-3
</code>
<br />
<br />
Using python (pip) to access https stuff, I got the error
<br />
<code>
Exception:
<br />
Traceback (most recent call last):
<br />
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
<br />
status = self.run(options, args)
<br />
File "/usr/lib/python2.7/dist-packages/pip/commands/search.py", line 40, in run
<br />
pypi_hits = self.search(query, index_url)
<br />
File "/usr/lib/python2.7/dist-packages/pip/commands/search.py", line 54, in search
<br />
hits = pypi.search({'name': query, 'summary': query}, 'or')
<br />
File "/usr/lib/python2.7/xmlrpclib.py", line 1233, in __call__
<br />
return self.__send(self.__name, args)
<br />
File "/usr/lib/python2.7/xmlrpclib.py", line 1591, in __request
<br />
verbose=self.__verbose
<br />
File "/usr/lib/python2.7/xmlrpclib.py", line 1273, in request
<br />
return self.single_request(host, handler, request_body, verbose)
<br />
File "/usr/lib/python2.7/xmlrpclib.py", line 1301, in single_request
<br />
self.send_content(h, request_body)
<br />
File "/usr/lib/python2.7/xmlrpclib.py", line 1448, in send_content
<br />
connection.endheaders(request_body)
<br />
File "/usr/lib/python2.7/httplib.py", line 997, in endheaders
<br />
self._send_output(message_body)
<br />
File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
<br />
self.send(msg)
<br />
File "/usr/lib/python2.7/httplib.py", line 812, in send
<br />
self.connect()
<br />
File "/usr/lib/python2.7/httplib.py", line 1212, in connect
<br />
server_hostname=server_hostname)
<br />
File "/usr/lib/python2.7/ssl.py", line 350, in wrap_socket
<br />
_context=self)
<br />
File "/usr/lib/python2.7/ssl.py", line 566, in __init__
<br />
self.do_handshake()
<br />
File "/usr/lib/python2.7/ssl.py", line 788, in do_handshake
<br />
self._sslobj.do_handshake()
<br />
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
<br />
</code>
<br />
<br />
Quick fix (can or cannot work... and not to be permanent as not
<br />
verifying the certificates is not very smart)
<br />
In your code, do:
<br />
<code>
<br />
import ssl
<br />
ssl._create_default_https_context = ssl._create_unverified_context
<br />
</code>
<br />
<br />
Permanent solution
<br />
<br />
<code>
<br />
rmdir /usr/local/share/ca-certificates (was blank)
<br />
ln -s /usr/share/ca-certificates/mozilla /usr/local/share/ca-certificates
<br />
sudo apt-get install ca-certificates # in case you don't have it already
<br />
sudo update-ca-certificates
<br />
<br />
# the new files will be sym-linked here:
<br />
/etc/ssl/certs
<br />
</code>
<br />
That fixed it for me at least.rikijpnhttp://www.blogger.com/profile/11964639797022470535noreply@blogger.com1