"no node found with given prefix" during cluster configuration


#1

I’m trying to setup a 3 node cluster (one master aggregator and two leaf nodes) in AWS running on CentOS 7.6. We use terraform instead of cloudformation, so using the Cloud instructions here wasn’t an option for me:

I chose r4.2xlarge for my instance sizes, which ends up providing too much memory for the free license I’m trying to test MemSQL with in our dev environment.

stderr: Error 2350: Leaf or aggregator node could not be added because you are using the MemSQL free license which has a total memory limit of 128.00 GB and after adding the node your total MemSQL memory capacity (used_cluster_capacity) would be 161.47 GB. To purchase more capacity, contact team@memsql.com. See https://docs.memsql.com/redir/memory-limit-error for instructions on how to set node memory limits to create a cluster with up to 128.00 GB of memory capacity.

I’m trying to do a comprehensive installation so I can limit the amount of memory per the instructions here:

Here are the steps I’ve done so far (from host 10.145.80.161):

  1. memsql-toolbox-config register-host --identity-file /home/centos/key.pem --host 10.145.80.161
  2. memsql-toolbox-config register-host --identity-file /home/centos/key.pem --host 10.145.80.197
  3. memsql-toolbox-config register-host --identity-file /home/centos/key.pem --host 10.145.80.179
  4. memsql-deploy install --all
  5. memsql-admin create-node --host 10.145.80.161 --password

I then tried to limit the memory of the master aggregator with this command:
memsql-admin update-config --set-global --key maximum_memory --value 30208 --memsql-id 84619EC4AFF18B21DE8B246E6250CD2119814C4A

This was the error I got:
no node found with given prefix

Here is the verbose output from the command:
https://pastebin.com/ZQ8yP2aG

Next, I followed the link provided in the error:

The memsql.cnf file exists and has the correct permissions:
[root@ip-10-145-80-161 6be4123d-7f86-436c-9c7e-d1c4ca7c71bc]# ls -l memsql.cnf
-rw-------. 1 memsql memsql 471 Apr 11 20:36 memsql.cnf

I also tried the ‘memsql-admin describe-node’ command and got the same error:
no node found with given prefix

Here is the verbose output of that command:
https://pastebin.com/yba2ngVW

Any help with this error would be greatly appreciated! Please let me know if you need any more details.


#2

The sum of the physical memory on the machines where you are running the MemSQL leaves has to be 128GB or less for the free version.


#3

Hi @hanson,

I changed it so I am now using four r4-xlarge nodes (1 master aggregator and three leaves). Each node has 30GB of memory, so total is around 120GB. I’m still getting that same error (“no node found with given prefix”) when I try this command:

memsql-admin bootstrap-aggregator --memsql-id BC1112D9519CE8B370C08EDB70C7B49FD302289D --license <license-number>

Verbose output is here:
https://pastebin.com/kKhMSMdN

Any idea why I’m still seeing this error?


#4

Hi @hanson,

Any idea what the error “no node found with given prefix” means and how to get past it? I need to get this cluster up and running ASAP for our analytics team to start testing MemSQL in our dev environment. The next step after that will be to purchase an enterprise license for our prod environment but we won’t be able to move forward with that until I can get them going in dev.

Thanks!


#5

Hi @hanson,

I just tried this again with a fresh set of EC2 instances and I’m getting the same error. What does “no node found with given prefix” mean? Here are the commands I ran:

[centos@ip-10-145-80-184 data]$ memsql-admin create-node --host 10.145.80.184 --password <secure-password> --datadir /data
Toolbox is about to perform the following actions on host 10.145.80.184:
  · Run 'memsqlctl create-node --datadir /data --password ●●●●●●'

Would you like to continue? [y/N]: y
+------------------------------------------+---------------+
|                MemSQL ID                 |     Host      |
+------------------------------------------+---------------+
| 26C1D20F68635EB27FA0623058B802AF9A8D50BC | 10.145.80.184 |
+------------------------------------------+---------------+
[centos@ip-10-145-80-184 data]$ memsql-admin bootstrap-aggregator --memsql-id 26C1D20F68635EB27FA0623058B802AF9A8D50BC --license <license-id>
no node found with given prefix