New Member
Posts: 14
Registered: ‎05-16-2018
Kudos: 3

What is HFQ exactly?

Hi,

 

Having trouble undestanding what HFQ is and does and does not do.

People on the forum tried asking for past couple years and none walked away with much clarity.

 

Could one of the Ubiquiti engineers please give clear black and white answers to following questions:

- Does HFQ allow bandwidth sharing/borrowing? i.e. Each host is able to use all bandwidth within allocated channel if unused by other hosts within subnet.
- Does HFQ dynamically adjust to number of hosts? i.e. Bandwidth keeps dynamically and fairly splitting within subnet as more or less devices are connected to wifi.

- How is HFQ different with PCQ? i.e. is HFQ a collection of dynamically changing FIFO queues like PCQ or is it something else? How does it work?

 

It is not possible to work out the above from the scape goat link that is being tossed around:

https://community.ubnt.com/t5/EdgeRouter/EdgeMAX-Advanced-queue-CLI-examples/m-p/1515779#U1515779

 

Overall support experience leaves much for concern to a potential customer that has not bought anything Ubiquiti as yet. While fairly responsive, email support has minimal value as all actual questions are being dodged. It's either an implied "i do not know and will not really help" or "go to the community or distributors". Community is also not much help in terms of QoS going by statistics of past few years.

Why not make a positive change?

Veteran Member
Posts: 7,822
Registered: ‎03-24-2016
Kudos: 2037
Solutions: 899

Re: What is HFQ exactly?

1 , 2 ) HFQ itself doesn't allow sharing/borrowing.  But when you use hfq as leaf queue under bandwidth limited branch, it does.

Example: 10 Mbit branch speed, and hfq speed set to 10Mb/s:  If one client is present/active, it gets full 10Mb/s speed, if 2 clients are active, each gets half of it.

 

3)  afaik , pcq is Mikrotik thingy.  I never managed to get to Mikrotik shell to see what tc config this leaves in place, but it does closely resemble hfq.   On ER, you can use tc command to see what is created under the hood: 

HFQ creates a per client leaf , giving  each client its own pfifo_fast queue type.(as memory serves me well)

 

 

 

 

New Member
Posts: 14
Registered: ‎05-16-2018
Kudos: 3

Re: What is HFQ exactly?

Much thanks 16again

 

I have no way to test if leaf pfifo_fast queues are being dynamically updated as devices come and leave the subnet.

Also have no way to test the way borrowing/sharing badwidth works with a few inactive and a few active devices.

HFQ does appear to be something similar to Mikrotik's PCQ.

 

Kindly requesting one of Ubiquiti engineers to clarify all uncertainties in above posts.

 

Bonus: kindly requesting Ubiquiti staff to create documentation that addresses questions above.

Veteran Member
Posts: 7,822
Registered: ‎03-24-2016
Kudos: 2037
Solutions: 899

Re: What is HFQ exactly?

For sure those per-client queues aren't created dynamically when clients enter.  For a /24 subnet, all 256 individual queues are created, and if there's no client using its queue, the queue sits just empty

This is script I use to look how ER QoS config translates into linux tc settings.  When I ran it on hfq config, I was overwhelmed by the amount of queues present

Spoiler

admin@ERX:/$ cat /config/scripts/tcshow
#!/bin/bash

if [ "$1" == "" ]; then
  echo "Usage: <$0 interface-name> "
  exit 1
fi
if [ `ip link show | grep -c $1` == "0" ]; then
  echo "Seems like $1 is not a valid interface name"
  echo "Usage: <tcshow interface-name> "
  exit 1
fi

echo "########## QDISC #############"
sudo tc -s qdisc show dev $1
echo "##"
echo "##"
echo "########## CLASS #############"
sudo tc -s class show dev $1
echo "##"
echo "##"
echo "########## FILTER #############"
sudo tc -s filter show dev $1
for i in `sudo tc -s filter show dev $1 | awk '{for(i=1;i<=NF;i++)if($i~/flowid/)print $(i+1)}'`
do
  if [ `sudo tc -s filter show dev $1 parent $i | grep -c filter` != "0" ]; then
    echo "####### nested FILTER #########"
    echo "       Filter(s) on $i"
    sudo tc -s filter show dev $1 parent $i
  fi
done
echo "##"
echo "##"

 

All it takes to test borrow is 2 PCs , an ER and a internet connection

New Member
Posts: 14
Registered: ‎05-16-2018
Kudos: 3

Re: What is HFQ exactly?

Assistance much appreciated 16again

Looks like HFQ is a bit of a dirty hack to be a pretend PCQ.
/22 subnet limit also makes sense now.
PCQ looks to be dynamic as there is a classification qualifier prior to fifo:
https://wiki.mikrotik.com/wiki/Manual:Queues_-_PCQ

As mentioned, I am not able to test as do not own Ubiquiti equipment. Nor do I own Mikrotik equipment at this time.

Question about borrowing still remains.
Waiting for Ubiquiti staff to confirm.
Veteran Member
Posts: 7,822
Registered: ‎03-24-2016
Kudos: 2037
Solutions: 899

Re: What is HFQ exactly?

Just played around with it:

Spoiler

set traffic-control advanced-queue filters match 106 attach-to 1
set traffic-control advanced-queue filters match 106 description MatchAll_IPv4
set traffic-control advanced-queue filters match 106 ether protocol ip
set traffic-control advanced-queue filters match 106 target 106
set traffic-control advanced-queue leaf queue 106 bandwidth 10mbit
set traffic-control advanced-queue leaf queue 106 ceiling 10mbit
set traffic-control advanced-queue leaf queue 106 parent 1
set traffic-control advanced-queue leaf queue 106 priority 0
set traffic-control advanced-queue leaf queue 106 queue-type MyHFQ
set traffic-control advanced-queue queue-type hfq MyHFQ host-identifier sip
set traffic-control advanced-queue queue-type hfq MyHFQ max-rate 9mbit
set traffic-control advanced-queue queue-type hfq MyHFQ subnet 10.x.y.64/30
set traffic-control advanced-queue root queue 1 attach-to eth0
set traffic-control advanced-queue root queue 1 bandwidth 11mbit

Note I used 9 , 10 and 11 mbit so from tc output , its clear where each setting ends up

Also , used /30 so only 4 client queues are created

 

 

From tcshow commands it's clear each IP gets it's own pfifo queue and htb rate limiter.

 this link explains filter,  which sends traffic to correct client queue

http://lartc.org/howto/lartc.adv-filter.hashing.html

 

Spoiler

 tcshow eth0
########## QDISC #############
qdisc htb 7000: root refcnt 2 r2q 10 default 0 direct_packets_stat 78 direct_qlen 1000
 Sent 44649 bytes 112 pkt (dropped 0, overlimits 55 requeues 0)
 backlog 0b 0p requeues 0
qdisc htb 7001: parent 7000:6a r2q 10 default 0 direct_packets_stat 0 direct_qlen 1000
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 9 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo 8019: parent 7001:400 limit 10p
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo 801a: parent 7001:401 limit 10p
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo 801b: parent 7001:402 limit 10p
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc pfifo 801c: parent 7001:403 limit 10p
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
##
##
########## CLASS #############
class htb 7000:6a parent 7000:1 leaf 7001: prio 0 rate 10Mbit ceil 10Mbit burst 1600b cburst 1600b
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8520bit 1pps backlog 0b 0p requeues 0
 lended: 34 borrowed: 0 giants: 0
 tokens: -1712 ctokens: -1712

class htb 7000:1 root rate 11Mbit ceil 11Mbit burst 1597b cburst 1597b
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8520bit 1pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 5191 ctokens: 5191

class htb 7001:1 root rate 1Gbit ceil 1Gbit burst 1375b cburst 1375b
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8520bit 1pps backlog 0b 0p requeues 0
 lended: 33 borrowed: 0 giants: 0
 tokens: 167 ctokens: 167

class htb 7001:400 parent 7001:1 leaf 8019: prio 0 rate 8bit ceil 10Mbit burst 274b cburst 1600b
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -1 ctokens: 20000

class htb 7001:401 parent 7001:1 leaf 801a: prio 0 rate 8bit ceil 10Mbit burst 274b cburst 1600b
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -1 ctokens: 20000

class htb 7001:402 parent 7001:1 leaf 801b: prio 0 rate 8bit ceil 10Mbit burst 274b cburst 1600b
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -1 ctokens: 20000

class htb 7001:403 parent 7001:1 leaf 801c: prio 0 rate 8bit ceil 10Mbit burst 274b cburst 1600b
 Sent 33245 bytes 34 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8520bit 1pps backlog 0b 0p requeues 0
 lended: 1 borrowed: 33 giants: 0
 tokens: -839450539 ctokens: -1826

##
##
########## FILTER #############
filter parent 7000: protocol ip pref 106 u32
filter parent 7000: protocol ip pref 106 u32 fh 800: ht divisor 1
filter parent 7000: protocol ip pref 106 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 7000:6a  (rule hit 34 success 34)
##
##

Note only .67 IP address has seen some traffic

 

Seems to me hfq and pcq both are (almost) the same

 

New Member
Posts: 14
Registered: ‎05-16-2018
Kudos: 3

Re: What is HFQ exactly?

[ Edited ]

Once again, thank you 16again

Please let me know if my undestanding summarised below is correct.

 

PCQ:

- packets are filtered by mangle marks before getting to fifo stage. Mangle marks are assigned dynamically on client logon.

- fifo queues are created dynamically via "address list" function as per https://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf

- bandwidth borrowing is setup by setting pcq-rate to total bandwidth of parent queue or to amount that is more than equal share

 

HFQ:

- no filter for packet selection/classification unless manually setup via cli

- pfifo_fast queues are created for every ip within subnet

- bandwidth borrowing = unclear if borrowing occurs if "Max Rate" is set to total bandwidth of parent queue = example with 60mbit hfq limit on page 52 of https://dl.ubnt.com/guides/edgemax/EdgeOS_UG.pdf hints that borrowing does occur = needs confirmation

 

My needs:

Create user group categories with own QoS rules. This may become possible once https://community.ubnt.com/t5/UniFi-Routing-Switching-Feature/Bandwidth-limit-by-VLAN-Network/idi-p/... is implemented.

VLAN assignment will occur on signon based on login details. "User groups" will be substituted for VLANs in terms of QoS rule segregation.

Question of how to configure borrowing between VLANs remains = HFQ as parent to VLAN queues?

 

edit = pcq summary correction

Veteran Member
Posts: 7,822
Registered: ‎03-24-2016
Kudos: 2037
Solutions: 899

Re: What is HFQ exactly?


MT - PCQ.   PCQ is just a queue like sfq, it needs htb above.
In MT world , selecting packets for QoS requires setting packet mark.  This should be done for each individual packet or connection, and is unrelated to client logon. (assigning a different vlan using 802.1x on client login can group clients in VLANs)
Address lists (=linux ipset) only have  use  in MT QoS world for packet marking.  Seems like lists were pretty new back in 2008, that's the only reason to mention them in presentation
Individual PCQ fifo queues are created for every ip within subnet. Nothing dynamic......but I have no cli MT access to confirm that with tc show output

In ER land, we can use same packet marking (firewall modify rules) and filter on those marks in QoS policy.  But it also allows filtering on adddress/port/dscp directly.
 
What you want is doable with link in post #1
https://community.ubnt.com/t5/EdgeRouter/EdgeMAX-Advanced-queue-CLI-examples/m-p/1515779#U1515779
Use advanced-queue, attach to root.Split all traffic in up and download , having it's own htb shaper. 
Download gets a per VLAN branch, using its HFQ. Down-vlans can borrow amongst each other, like down-clients in same vlan can

And similar hierarchy is used for  upload
New Member
Posts: 14
Registered: ‎05-16-2018
Kudos: 3

Re: What is HFQ exactly?

[ Edited ]

Thanks 16again

Help is very much of value

 

Remains to confirm for PCQ:

FIFO per every ip within subnet = most recent update on PCQ dated 2013 page 36 onwards https://mum.mikrotik.com/presentations/HR13/valens.pdf

Dynamic is in a sense that "address list" changes packet marking policy on login -> once in the right PCQ, further FIFO separation is done via matching src/dst ip and ports.

 

 

Summary of required solution:

RADIUS controlled VLAN assignment -> inter-VLAN routing -> per VLAN QoS -> inter-VLAN borrowing -> Traffic Prioritisation -> Shot at Dynamic to optimise performace

 

- RADIUS controlled VLAN assignment = good to go

- inter-VLAN routing = VLAN separation is done as a hack because EdgeMax does not support user groups. Need is for all VLANs to be able to talk to each other transparently as if it was a single LAN

- per VLAN QoS = good to go

- inter VLAN borrowing = unsure if VLANS will borrow amongst each other with HFQ set per VLAN

- Traffic Prioritisation = how to setup prioritisation as per pages 29-30 of https://mum.mikrotik.com/presentations/US08/janism.pdf

- Shot at Dynamic = Is there a way to run a script on login that deletes unused fifo queues based on refreshing arp table or similar? Unsure how large amounts of fifo affect performance, assuming memory is taken up.

 

Further questions:

Is there a way to do the same thing as PCQ = i.e. separate into different HFQs via packet marking and DPI?

How much more CPU/memory efficient is doing away without DPI via VLAN separation?

 

Edit:

Based on further research, what I really need is PacketFence and role based QoS/firewall rules.

EdgeSwitch has support.

Unifi does not have support as yet = https://community.ubnt.com/t5/UniFi-Feature-Requests/PacketFence-NAC-support/idc-p/2134999#M12507

Another option is to run OpenWRT on Unifi

 

Is Unifi support for PacketFence necessary in a EdgeSwitch + Unifi setup ?

Veteran Member
Posts: 7,822
Registered: ‎03-24-2016
Kudos: 2037
Solutions: 899

Re: What is HFQ exactly?

For sure, move the inter-VLAN routing to a L3 switch. 

This reduces CPU on the router.  (Note MT has clever mangle +fasttrack combo, giving option  to leave inter-VLAN traffic untouched by QoS)